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Responding to a failed test that you ran is much easier than responding to a 
bug report from an unhappy user. 

Other programmers will respect your projects more if you include some 
initial tests. They'll feel more comfortable experimenting with your code 
and be more willing to work with you on projects. If you want to contribute 
to a project that other programmers are working on, you'll be expected to 
show that your code passes existing tests and you'll usually be expected 
to write tests for any new behavior you introduce to the project. 

Play around with tests to become familiar with the process of testing 
your code. Write tests for the most critical behaviors of your functions and 
classes, but don't aim for full coverage in early projects unless you have a 
specific reason to do so. 
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PART H 


PROJECTS 


Congratulations! You now know enough about Python 
to start building interactive and meaningful projects. 
Creating your own projects will teach you new skills 
and solidify your understanding of the concepts intro- 
duced in Part I. 


Part II contains three kinds of projects, and you can choose to do any 
or all of these projects in whichever order you like. Here's a brief descrip- 
tion of each project to help you decide which to dig into first. 


Alien Invasion: Making a Game with Python 


In the Alien Invasion project (Chapters 12, 13, and 14), you'll use the 
Pygame package to develop a 2D game. The goal of the game is to shoot 
down a fleet of aliens as they drop down the screen, in levels that increase 
in speed and difficulty. At the end of the project, you'll have learned skills 
that will enable you to develop your own 2D games in Pygame. 


Data Visualization 


The Data Visualization projects start in Chapter 15, where you'll learn to 
generate data and create a series of functional and beautiful visualizations 
of that data using Matplotlib and Plotly. Chapter 16 teaches you to access 
data from online sources and feed it into a visualization package to cre- 
ate plots of weather data and a map of global earthquake activity. Finally, 
Chapter 17 shows you how to write a program to automatically download 
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and visualize data. Learning to make visualizations allows you to explore 
the field of data science, which is one of the highest-demand areas of pro- 
gramming today. 


Web Applications 


Part Il: Projects 


In the Web Application project (Chapters 18, 19, and 20), you'll use the 
Django package to create a simple web application that allows users to keep 
a journal about different topics they've been learning about. Users will cre- 
ate an account with a username and password, enter a topic, and then make 
entries about what they're learning. You'll also deploy your app to a remote 
server so anyone in the world can access it. 

After completing this project, you'll be able to start building your own 
simple web applications, and you'll be ready to delve into more thorough 
resources on building applications with Django. 


A SHIP THAT FIRES BULLETS 


Let's build a game called Alien Invasion! 
We'll use Pygame, a collection of fun, pow- 


erful Python modules that manage graphics, 
animation, and even sound, making it easier 

for you to build sophisticated games. With Pygame 

handling tasks like drawing images to the screen, you 


can focus on the higher-level logic of game dynamics. 


In this chapter, you'll set up Pygame and then create a rocket ship that 
moves right and left and fires bullets in response to player input. In the next 
two chapters, you'll create a fleet of aliens to destroy, and then continue to 
refine the game by setting limits on the number of ships you can use and 
adding a scoreboard. 

While building this game, you'll also learn how to manage large proj- 
ects that span multiple files. We'll refactor a lot of code and manage file 
contents to organize the project and make the code efficient. 
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Making games is an ideal way to have fun while learning a language. It’s 
deeply satisfying to play a game you wrote, and writing a simple game will 
teach you a lot about how professionals develop games. As you work through 
this chapter, enter and run the code to identify how each code block contrib- 
utes to overall gameplay. Experiment with different values and settings to 
better understand how to refine interactions in your games. 


Alien Invasion spans a number of different files, so make a new alien_invasion 
folder on your system. Be sure to save all files for the project to this folder so your 
import statements will work correctly. 

Also, if you feel comfortable using version control, you might want to use it 
for this project. If you haven't used version control before, see Appendix D for an 
overview. 


Planning Your Project 


When you're building a large project, it’s important to prepare a plan before 
you begin to write code. Your plan will keep you focused and make it more 
likely that you'll complete the project. 

Let’s write a description of the general gameplay. Although the following 
description doesn’t cover every detail of Alien Invasion, it provides a clear idea 
of how to start building the game: 


In Alien Invasion, the player controls a rocket ship that appears 

at the bottom center of the screen. The player can move the ship 
right and left using the arrow keys and shoot bullets using the 
spacebar. When the game begins, a fleet of aliens fills the sky 

and moves across and down the screen. The player shoots and 
destroys the aliens. If the player destroys all the aliens, a new fleet 
appears that moves faster than the previous fleet. If any alien hits 
the player’s ship or reaches the bottom of the screen, the player 
loses a ship. If the player loses three ships, the game ends. 


For the first development phase, we’ll make a ship that can move right 
and left when the player presses the arrow keys and fire bullets when the 
player presses the spacebar. After setting up this behavior, we can create the 
aliens and refine the gameplay. 


Installing Pygame 


Chapter 12 


Before you begin coding, install Pygame. We’ll do this the same way we 
installed pytest in Chapter 11: with pip. If you skipped Chapter 11 or need a 
refresher on pip, see “Installing pytest with pip" on page 210. 

To install Pygame, enter the following command at a terminal prompt: 


$ python -m pip install --user pygame 


If you use a command other than python to run programs or start a ter- 
minal session, such as python3, make sure you use that command instead. 


Starting the Game Project 


We’ll begin building the game by creating an empty Pygame window. Later, 
we'll draw the game elements, such as the ship and the aliens, on this win- 
dow. We'll also make our game respond to user input, set the background 
color, and load a ship image. 


Creating a Pygame Window and Responding to User Input 


We'll make an empty Pygame window by creating a class to represent the 
game. In your text editor, create a new file and save it as alien, invasion.py; 
then enter the following: 


alien import sys 


_invasion.py 
import pygame 


class AlienInvasion: 
"""Overall class to manage game assets and behavior.""" 
def init (self): 
"""Initialize the game, and create game resources. 
e pygame.init() 


e self.screen - pygame.display.set mode((1200, 800)) 
pygame.display.set caption("Alien Invasion") 


def run game(self): 
"""Start the main loop for the game. 
e while True: 
# Watch for keyboard and mouse events. 
for event in pygame.event.get(): 
if event.type -- pygame.QUIT: 
sys.exit() 


o6 


# Make the most recently drawn screen visible. 
[9] pygame.display.flip() 


if name == ' main 
# Make a game instance, and run the game. 
ai - AlienInvasion() 
ai.run game() 


First, we import the sys and pygame modules. The pygame module con- 
tains the functionality we need to make a game. We'll use tools in the sys 
module to exit the game when the player quits. 

Alien Invasion starts as a class called AlienInvasion. In the — init () 
method, the pygame. init() function initializes the background settings that 
Pygame needs to work properly €. Then we call pygame.display.set mode() 
to create a display window @, on which we'll draw all the game's graphical 
elements. The argument (1200, 800) is a tuple that defines the dimensions 
of the game window, which will be 1,200 pixels wide by 800 pixels high. 
(You can adjust these values depending on your display size.) We assign this 
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display window to the attribute self.screen, so it will be available in all meth- 
ods in the class. 

The object we assigned to self.screen is called a surface. A surface in 
Pygame is a part of the screen where a game element can be displayed. Each 
element in the game, like an alien or a ship, is its own surface. The surface 
returned by display.set_mode() represents the entire game window. When we 
activate the game’s animation loop, this surface will be redrawn on every 
pass through the loop, so it can be updated with any changes triggered by 
user input. 

The game is controlled by the run_game() method. This method contains 
awhile loop © that runs continually. The while loop contains an event loop 
and code that manages screen updates. An event is an action that the user per- 
forms while playing the game, such as pressing a key or moving the mouse. To 
make our program respond to events, we write an event loop to listen for events 
and perform appropriate tasks depending on the kinds of events that occur. 
The for loop @ nested inside the while loop is an event loop. 

To access the events that Pygame detects, we'll use the pygame.event.get() 
function. This function returns a list of events that have taken place since 
the last time this function was called. Any keyboard or mouse event will 
cause this for loop to run. Inside the loop, we'll write a series of if state- 
ments to detect and respond to specific events. For example, when the 
player clicks the game window's close button, a pygame.QUIT event is detected 
and we call sys.exit() to exit the game 6. 

The call to pygame.display.flip() O tells Pygame to make the most 
recently drawn screen visible. In this case, it simply draws an empty screen 
on each pass through the while loop, erasing the old screen so only the new 
screen is visible. When we move the game elements around, pygame.display 
.flip() continually updates the display to show the new positions of game 
elements and hide the old ones, creating the illusion of smooth movement. 

At the end of the file, we create an instance of the game and then call 
run game(). We place run game() in an if block that only runs if the file is 
called directly. When you run this alien invasion.py file, you should see an 
empty Pygame window. 


Controlling the Frame Rate 


Ideally, games should run at the same speed, or frame rate, on all systems. 
Controlling the frame rate of a game that can run on multiple systems is 
a complex issue, but Pygame offers a relatively simple way to accomplish 
this goal. We'll make a clock, and ensure the clock ticks once on each pass 
through the main loop. Anytime the loop processes faster than the rate we 
define, Pygame will calculate the correct amount of time to pause so that 
the game runs at a consistent rate. 
We'll define the clock in the init () method: 


T 


self.clock = pygame.time.Clock() 


After initializing pygame, we create an instance of the class Clock, from 
the pygame.time module. Then we'll make the clock tick at the end of the 
while loop in run game(): 


self.clock.tick(60) 


The tick() method takes one argument: the frame rate for the game. 
Here I'm using a value of 60, so Pygame will do its best to make the loop 
run exactly 60 times per second. 


Pygame's clock should help the game run consistently on most systems. If it makes the 
game run less consistently on your system, you can try different values for the frame 
rale. If you can't find a good frame rate om your system, you can leave the clock out 
entirely and adjust the game's settings so it runs well on your system. 


Setting the Background Color 


Pygame creates a black screen by default, but that's boring. Let's set a differ- 
ent background color. We'll do this at the end of the — init () method. 


alien 
_invasion.py 
# Set the background color. 
o self.bg color - (230, 230, 230) 
# Redraw the screen during each pass through the loop. 
e self.screen.fill(self.bg color) 


Colors in Pygame are specified as RGB colors: a mix of red, green, and 
blue. Each color value can range from 0 to 255. The color value (255, 0, 0) 
is red, (0, 255, 0) is green, and (0, 0, 255) is blue. You can mix different 
RGB values to create up to 16 million colors. The color value (230, 230, 230) 
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mixes equal amounts of red, blue, and green, which produces a light gray 
background color. We assign this color to self.bg color 6». 

We fill the screen with the background color using the fill() method 6, 
which acts on a surface and takes only one argument: a color. 


Creating a Settings Class 


Each time we introduce new functionality into the game, we'll typically 
create some new settings as well. Instead of adding settings throughout the 
code, let's write a module called settings that contains a class called Settings 
to store all these values in one place. This approach allows us to work with 
just one settings object anytime we need to access an individual setting. 
This also makes it easier to modify the game's appearance and behavior as 
our project grows. To modify the game, we'll change the relevant values in 
settings.py, which we'll create next, instead of searching for different settings 
throughout the project. 

Create a new file named settings.py inside your alien, invasion folder, and 
add this initial Settings class: 


settings.py class Settings: 
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A class to store all settings for Alien Invasion. 


def init (self): 
"""Initialize the game's settings. 
# Screen settings 
self.screen width - 1200 
self.screen height - 800 
self.bg color - (230, 230, 230) 


To make an instance of Settings in the project and use it to access our 
settings, we need to modify alien invasion.py as follows: 


from settings import Settings 


J5 \ 7 
self.clock = 


self.settings = Settings() . 


self.screen = pygame.display.set_mode( 
(self.settings.screen width, self.settings.screen_ height) ) 


e self.screen.fill(self.settings.bg color) 


ecently drawn screen visible 


We import Settings into the main program file. Then we create an 
instance of Settings and assign it to self.settings 69, after making the call 
to pygane.init(). When we create a screen , we use the screen width and 
screen height attributes of self.settings, and then we use self.settings to 
access the background color when filling the screen ® as well. 

When you run alien, invasion.py now you won't yet see any changes, 
because all we've done is move the settings we were already using elsewhere. 
Now we're ready to start adding new elements to the screen. 


Adding the Ship Image 


Let's add the ship to our game. To draw the player's ship on the screen, 
we'll load an image and then use the Pygame blit() method to draw the 
image. 

When you're choosing artwork for your games, be sure to pay atten- 
tion to licensing. The safest and cheapest way to start is to use freely 
licensed graphics that you can use and modify, from a website like https:// 
opengameart.org. 

You can use almost any type of image file in your game, but it's easiest 
when you use a bitmap (.bmp) file because Pygame loads bitmaps by default. 
Although you can configure Pygame to use other file types, some file types 
depend on certain image libraries that must be installed on your computer. 
Most images you'll find are in jpg or .png formats, but you can convert them 
to bitmaps using tools like Photoshop, GIMP, and Paint. 

Pay particular attention to the background color in your chosen 
image. Try to find a file with a transparent or solid background that you 
can replace with any background color, using an image editor. Your games 
will look best if the image's background color matches your game's back- 
ground color. Alternatively, you can match your game's background to the 
image's background. 

For Alien Invasion, you can use the file ship.bmp (Figure 12-1), which is 
available in this book's resources at https://ehmatthes.github.io/pcc_3e. The 
file's background color matches the settings we're using in this project. 
Make a folder called images inside your main alien invasion project folder. 
Save the file ship.bmp in the images folder. 
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Figure 12-1: The ship for Alien Invasion 


Creating the Ship Class 


After choosing an image for the ship, we need to display it on the screen. To 
use our ship, we’ll create a new ship module that will contain the class Ship. 
This class will manage most of the behavior of the player’s ship: 


import pygame 


class Ship: 
"""A class to manage the ship.""" 
def init (self, ai game): 
"""Initialize the ship and set its starting position. 
self.screen - ai game.screen 
self.screen rect - ai game.screen.get rect() 


# Load the ship image and get its rect. 
self.image - pygame.image.load('images/ship.bmp') 
self.rect - self.image.get rect() 


# Start each new ship at the bottom center of the screen. 
self.rect.midbottom = self.screen rect.midbottom 


def blitme(self): 
"""Draw the ship at its current location. 
self.screen.blit(self.image, self.rect) 


Pygame is efficient because it lets you treat all game elements like rect- 
angles (rects), even if they're not exactly shaped like rectangles. Treating an 
element as a rectangle is efficient because rectangles are simple geometric 
shapes. When Pygame needs to figure out whether two game elements have 
collided, for example, it can do this more quickly if it treats each object as a 
rectangle. This approach usually works well enough that no one playing the 
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game will notice that we’re not working with the exact shape of each game 
element. We’ll treat the ship and the screen as rectangles in this class. 

We import the pygame module before defining the class. The — init () 
method of Ship takes two parameters: the self reference and a reference to 
the current instance of the AlienInvasion class. This will give Ship access to 
all the game resources defined in AlienInvasion. We then assign the screen 
to an attribute of Ship @, so we can access it easily in all the methods in this 
class. We access the screen’s rect attribute using the get_rect() method and 
assign it to self.screen_rect @. Doing so allows us to place the ship in the 
correct location on the screen. 

To load the image, we call pygame. image. load() © and give it the location 
of our ship image. This function returns a surface representing the ship, 
which we assign to self.image. When the image is loaded, we call get_rect() to 
access the ship surface’s rect attribute so we can later use it to place the ship. 

When you're working with a rect object, you can use the x- and 
y-coordinates of the top, bottom, left, and right edges of the rectangle, as 
well as the center, to place the object. You can set any of these values to 
establish the current position of the rect. When you're centering a game 
element, work with the center, centerx, or centery attributes of a rect. When 
you're working at an edge of the screen, work with the top, bottom, left, or 
right attributes. There are also attributes that combine these properties, 
such as midbottom, midtop, midleft, and midright. When you're adjusting the 
horizontal or vertical placement of the rect, you can just use the x and y 
attributes, which are the x- and y-coordinates of its top-left corner. These 
attributes spare you from having to do calculations that game developers 
formerly had to do manually, and you'll use them often. 


In Pygame, the origin (0, 0) is at the top-left corner of the screen, and coordinates 
increase as you go down and to the right. On a 1200x800 screen, the origin is at the 
top-left corner, and the bottom-right corner has the coordinates (1200, 800). These 
coordinates refer to the game window, not the physical screen. 


We'll position the ship at the bottom center of the screen. To do so, 
make the value of self.rect.midbottom match the midbottom attribute of the 
screen's rect @. Pygame uses these rect attributes to position the ship image 
so it's centered horizontally and aligned with the bottom of the screen. 

Finally, we define the blitme() method ©, which draws the image to the 
screen at the position specified by self.rect. 


Drawing the Ship to the Screen 


Now let's update alien, invasion.py so it creates a ship and calls the ship’s 
blitme() method: 


from ship import Ship ! 
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det run game(seit): 


scree! 


l(self.se 


self.ship. blitme() 


seiT.screen. 


We import Ship and then make an instance of Ship after the screen has 
been created 6. The call to Ship() requires one argument: an instance 
of AlienInvasion. The self argument here refers to the current instance of 
AlienInvasion. This is the parameter that gives Ship access to the game's 
resources, such as the screen object. We assign this Ship instance to self.ship. 

After filling the background, we draw the ship on the screen by calling 
ship.blitme(), so the ship appears on top of the background 6. 

When you run alien, invasion.py now, you should see an empty game 
screen with the rocket ship sitting at the bottom center, as shown in 
Figure 12-2. 
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Figure 12-2: Alien Invasion with the ship at the bottom center of the screen 


Refactoring: The _check_events() and update screen() Methods 


alien 
_invasion.py 


e 


e 
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In large projects, you'll often refactor code you've written before adding 
more code. Refactoring simplifies the structure of the code you've already 
written, making it easier to build on. In this section, we'll break the run game() 
method, which is getting lengthy, into two helper methods. A helper method 
does work inside a class but isn't meant to be used by code outside the class. 
In Python, a single leading underscore indicates a helper method. 


The check events() Method 


We'll move the code that manages events to a separate method called 
check events(). This will simplify run game() and isolate the event manage- 
ment loop. Isolating the event loop allows you to manage events separately 
from other aspects of the game, such as updating the screen. 

Here's the AlienInvasion class with the new check events() method, 
which only affects the code in run game(): 


self. check events() 


def check events(self): 
"""Respond to keypresses and mouse events. 


We make a new check events() method and move the lines that check 
whether the player has clicked to close the window into this new method. 

To call a method from within a class, use dot notation with the variable 
self and the name of the method 6. We call the method from inside the 
while loop in run game(). 


The vpdate screen() Method 


To further simplify run gane(), we'll move the code for updating the screen 
to a separate method called update screen(): 


self. update screen() 
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def update screen(self): 
"""Update images on the screen, and flip to the new screen. 


Self. 


We moved the code that draws the background and the ship and flips 
the screen to update screen(). Now the body of the main loop in run game() 
is much simpler. It's easy to see that we're looking for new events, updating 
the screen, and ticking the clock on each pass through the loop. 

If you've already built a number of games, you'll probably start out by 
breaking your code into methods like these. But if you've never tackled a 
project like this, you probably won't know exactly how to structure your 
code at first. This approach gives you an idea of a realistic development pro- 
cess: you start out writing your code as simply as possible, and then refactor 
it as your project becomes more complex. 

Now that we've restructured the code to make it easier to add to, we can 
work on the dynamic aspects of the game! 


TRY IT YOURSELF 


12-1. Blue Sky: Make a Pygame window with a blue background. 


12-2. Game Character: Find a bitmap image of a game character you like or 
convert an image to a bitmap. Make a class that draws the character at the 
center of the screen, then match the background color of the image to the back- 
ground color of the screen or vice versa. 


Piloting the Ship 
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Next, we'll give the player the ability to move the ship right and left. We'll 
write code that responds when the player presses the right or left arrow key. 
We'll focus first on movement to the right, and then we'll apply the same 
principles to control movement to the left. As we add this code, you'll learn 
how to control the movement of images on the screen and respond to user 
input. 


Responding to a Keypress 


Whenever the player presses a key, that keypress is registered in Pygame as 
an event. Each event is picked up by the pygame.event.get() method. We need 
to specify in our. check events() method what kinds of events we want the 
game to check for. Each keypress is registered as a KEYDOWN event. 
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When Pygame detects a KEYDOWN event, we need to check whether the 
key that was pressed is one that triggers a certain action. For example, if the 
player presses the right arrow key, we want to increase the ship’s rect.x value 
to move the ship to the right: 


elif event.type == pygame.KEYDOWN: 
if event.key == pygame.K RIGHT: 
# Move the ship to the right. 
self.ship.rect.x += 1 


Inside check events() we add an elif block to the event loop, to respond 
when Pygame detects a KEYDOWN event €. We check whether the key pressed, 
event.key, is the right arrow key 9. The right arrow key is represented by 
pygame.K RIGHT. If the right arrow key was pressed, we move the ship to the 
right by increasing the value of self.ship.rect.x by 1 6. 

When you run alien, invasion.py now, the ship should move to the right 
one pixel every time you press the right arrow key. That's a start, but it's not 
an efficient way to control the ship. Let's improve this control by allowing 
continuous movement. 


Allowing Continuous Movement 


When the player holds down the right arrow key, we want the ship to continue 
moving right until the player releases the key. We'll have the game detect a 
pygame.KEYUP event so we'll know when the right arrow key is released; then 
we'll use the KEYDOWN and KEYUP events together with a flag called moving right 
to implement continuous motion. 

When the moving right flag is False, the ship will be motionless. When 
the player presses the right arrow key, we'll set the flag to True, and when the 
player releases the key, we'll set the flag to False again. 

The Ship class controls all attributes of the ship, so we'll give it an attri- 
bute called moving right and an update() method to check the status of the 
moving right flag. The update() method will change the position of the ship if 
the flag is set to True. We'll call this method once on each pass through the 
while loop to update the position of the ship. 

Here are the changes to Ship: 
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# Movement flag; start with a ship that's not moving. 
e self.moving right - False 


e def update(self): 
"""Update the ship's position based on the movement flag. 
if self.moving right: 
self.rect.x 4- 1 


We add a self.moving right attribute in the _ init () method and set it 
to False initially €. Then we add update(), which moves the ship right if the 
flag is True 6. The update() method will be called from outside the class, so 
it's not considered a helper method. 

Now we need to modify check events() so that moving right is set to True 
when the right arrow key is pressed and False when the key is released: 
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e self.ship.moving right = True 
e elif event.type -- pygame.KEYUP: 
if event.key -- pygame.K RIGHT: 


self.ship.moving right - False 


Here, we modify how the game responds when the player presses the 
right arrow key: instead of changing the ship's position directly, we merely 
set moving right to True €. Then we add a new elif block, which responds to 
KEYUP events 6. When the player releases the right arrow key (K RIGHT), we 
set moving right to False. 

Next, we modify the while loop in run game() so it calls the ship's update() 
method on each pass through the loop: 


alien invasion.py def ru 


self. ship.update() 7 


ICK.T1C 


The ship’s position will be updated after we’ve checked for keyboard 
events and before we update the screen. This allows the ship’s position to be 
updated in response to player input and ensures the updated position will 
be used when drawing the ship to the screen. 

When you run alien_invasion.py and hold down the right arrow key, the 
ship should move continuously to the right until you release the key. 
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Moving Both Left and Right 


Now that the ship can move continuously to the right, adding movement 
to the left is straightforward. Again, we’ll modify the Ship class and the 


_check_events() method. Here are the relevant changes to init () and 
update() in Ship: 


ship.py def init (se 
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# Movement flags; start with a ship that's not moving. 


self.moving left i False 


f update(self): 
"nn 


Update the ship's position based on movement flags.""" 
seit Bei x i 1 

if self.moving left: 

self.rect.x -= 1 


In init (), we add a self.moving left flag. In update(), we use two 
separate if blocks, rather than an elif, to allow the ship's rect.x value to be 
increased and then decreased when both arrow keys are held down. This 
results in the ship standing still. If we used elif for motion to the left, the 
right arrow key would always have priority. Using two if blocks makes the 
movements more accurate when the player might momentarily hold down 
both keys when changing directions. 

We have to make two additions to check events(): 


elif event.key == pygame.K LEFT: l 
self.ship.moving_left = True 


elif event.key == pygame.K_LEFT: 
self.ship.moving_left = False 


If a KEYDOWN event occurs for the K_LEFT key, we set moving_left to True. If a 
KEYUP event occurs for the K_LEFT key, we set moving_left to False. We can use 
elif blocks here because each event is connected to only one key. If the player 
presses both keys at once, two separate events will be detected. 

When you run alien_invasion.py now, you should be able to move the 


ship continuously to the right and left. If you hold down both keys, the ship 
should stop moving. 
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Next, we'll further refine the ship's movement. Let's adjust the ship's 
speed and limit how far the ship can move so it can't disappear off the sides 
of the screen. 


Adjusting the Ship's Speed 


Currently, the ship moves one pixel per cycle through the while loop, but we 
can take finer control of the ship's speed by adding a ship speed attribute to 

the Settings class. We'll use this attribute to determine how far to move the 

ship on each pass through the loop. Here's the new attribute in settings.py: 


# Ship settings 
self.ship speed - 1.5 


We set the initial value of ship speed to 1.5. When the ship moves now, 
its position is adjusted by 1.5 pixels (rather than 1 pixel) on each pass 
through the loop. 

We're using a float for the speed setting to give us finer control of the 
ship's speed when we increase the tempo of the game later on. However, 
rect attributes such as x store only integer values, so we need to make some 
modifications to Ship: 


class Ship: 
"mmm 


A class to manage the ship."" 


" 


init (self, ai game): 
Initialize the st 


"n 


Hp and set its starting position." 


self.screen - ai game.screen 
self.settings - ai game.settings 
--snip-- 


# Start each new ship at the bottom center of 


self.rect.midbottom - self.screen rect.midbottom 


the screen. 


# Store a float for the ship's exact horizontal position. 
self.x - float(self.rect.x) 


# Movement flags; start with a ship that's not moving. 


self.moving right - False 
self.moving left - False 
def update(self): 
"""Update the ship's position based on movement flags.""" 


# Update the ship's x value, not the rect. 
if self.moving right: 
self.x += self.settings.ship speed 


ship.py 


self.x as self.settings.ship speed 


# Update rect object from self.x. 
self.rect.x = self.x 


We create a settings attribute for Ship, so we can use it in update() 6. 
Because we’re adjusting the position of the ship by fractions of a pixel, we 
need to assign the position to a variable that can have a float assigned to it. 
You can use a float to set an attribute of a rect, but the rect will only keep 
the integer portion of that value. To keep track of the ship’s position accu- 
rately, we define a new self.x @. We use the float() function to convert the 
value of self.rect.x to a float and assign this value to self.x. 

Now when we change the ship’s position in update(), the value of self.x 
is adjusted by the amount stored in settings.ship speed ©. After self.x 
has been updated, we use the new value to update self.rect.x, which con- 
trols the position of the ship O. Only the integer portion of self.x will be 
assigned to self.rect.x, but that's fine for displaying the ship. 

Now we can change the value of ship speed, and any value greater than 1 
will make the ship move faster. This will help make the ship respond 
quickly enough to shoot down aliens, and it will let us change the tempo 
of the game as the player progresses in gameplay. 


limiting the Ship's Range 

At this point, the ship will disappear off either edge of the screen if you 
hold down an arrow key long enough. Let's correct this so the ship stops 
moving when it reaches the screen's edge. We do this by modifying the 
update() method in Ship: 


if pen „roving right a and self. rect. tonight < self.screen_rect.right: 


if sell moving. left and self. rect. left > 0: 


This code checks the position of the ship before changing the value of 
self.x. The code self.rect.right returns the x-coordinate of the right edge 
of the ship’s rect. If this value is less than the value returned by self.screen 

_rect.right, the ship hasn't reached the right edge of the screen €. The same 
goes for the left edge: if the value of the left side of the rect is greater than 0, 
the ship hasn't reached the left edge of the screen 6. This ensures the ship 
is within these bounds before adjusting the value of self.x. 
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When you run alien_invasion.py now, the ship should stop moving at 
either edge of the screen. This is pretty cool; all we’ve done is add a con- 
ditional test in an if statement, but it feels like the ship hits a wall or force 
field at either edge of the screen! 


Refactoring check events() 


The check events() method will increase in length as we continue to 
develop the game, so let's break | check events() into two separate methods: 
one that handles KEYDOWN events and another that handles KEYUP events: 


alien invasion.py def check events( 


self. check keydown events(event) 
self. check keyup events(event) 
def check keydown events(self, event): 
"""Respond to keypresses.""" 


S.A RLU 


def check keyup events(self, event): 
Respond to key releases.""" 


We make two new helper methods: check keydown events() and check 
_keyup_events(). Each needs a self parameter and an event parameter. The 
bodies of these two methods are copied from check events(), and we've 
replaced the old code with calls to the new methods. The check events() 
method is simpler now with this cleaner code structure, which will make it 
easier to develop further responses to player input. 


Pressing Q to Quit 


Now that we're responding to keypresses efficiently, we can add another 

way to quit the game. It gets tedious to click the X at the top of the game 
window to end the game every time you test a new feature, so we'll add a 
keyboard shortcut to end the game when the player presses Q: 


alien invasion.py def check keydown events(self, 
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elif event.key == pygame.K q:- 
sys.exit() 


In check keydown events(), we add a new block that ends the game when 
the player presses Q. Now, when testing, you can press O to close the game 
instead of using your cursor to close the window. 


Running the Game in Fullscreen Mode 


Pygame has a fullscreen mode that you might like better than running 
the game in a regular window. Some games look better in fullscreen 
mode, and on some systems, the game may perform better overall in 
fullscreen mode. 

To run the game in fullscreen mode, make the following changes in 
. init (): 


self.screen = pygame.display.set_mode((0, 0), pygame.FULLSCREEN) 
self.settings.screen width - self.screen.get rect().width 
self.settings.screen height - self.screen.get rect().height 


When creating the screen surface, we pass a size of (0, 0) and the 
parameter pygame.FULLSCREEN 6.. This tells Pygame to figure out a window 
size that will fill the screen. Because we don't know the width and height of 
the screen ahead of time, we update these settings after the screen is cre- 
ated 9. We use the width and height attributes of the screen's rect to update 
the settings object. 

If you like how the game looks or behaves in fullscreen mode, keep 
these settings. If you liked the game better in its own window, you can 
revert back to the original approach where we set a specific screen size for 
the game. 


Make sure you can quit by pressing Q before running the game in fullscreen mode; 
Pygame offers no default way to quit a game while in fullscreen mode. 


A Quick Recap 


In the next section, we'll add the ability to shoot bullets, which involves add- 
ing a new file called bullet.py and making some modifications to some of the 
files we're already using. Right now, we have three files containing a num- 
ber of classes and methods. To be clear about how the project is organized, 
let's review each of these files before adding more functionality. 
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alien_invasion.py 


The main file, alien_invasion.py, contains the AlienInvasion class. This class 
creates a number of important attributes used throughout the game: the 
settings are assigned to settings, the main display surface is assigned to 
screen, and a ship instance is created in this file as well. The main loop of 
the game, a while loop, is also stored in this module. The while loop calls 
_check_events(), ship.update(), and update screen(). It also ticks the clock on 
each pass through the loop. 

The check events() method detects relevant events, such as keypresses 
and releases, and processes each of these types of events through the meth- 
ods check keydown events() and check keyup events(). For now, these methods 
manage the ship's movement. The AlienInvasion class also contains update 
_screen(), which redraws the screen on each pass through the main loop. 

The alien, invasion.py file is the only file you need to run when you want 
to play Alien Invasion. The other files, settings.py and ship.py, contain code 
that is imported into this file. 


settings.py 


The settings.py file contains the Settings class. This class only has an — init () 
method, which initializes attributes controlling the game's appearance and 
the ship's speed. 


ship.py 

The ship.py file contains the Ship class. The Ship class has an init () method, 
an update() method to manage the ship’s position, and a blitme() method to 
draw the ship to the screen. The image of the ship is stored in ship.bmp, which 
is in the images folder. 


TRY IT YOURSELF 


12-3. Pygame Documentation: We're far enough into the game now that you 
might want to look at some of the Pygame documentation. The Pygame home 
page is at hitps://pygame.org, and the home page for the documentation is at 
https://pygame.org/docs. Just skim the documentation for now. You won't need 
it to complete this project, but it will help if you want to modify Alien Invasion or 
make your own game afterward. 


12-4. Rocket: Make a game that begins with a rocket in the center of the screen. 


Allow the player to move the rocket up, down, left, or right using the four arrow 


keys. Make sure the rocket never moves beyond any edge of the screen. 

12-5. Keys: Make a Pygame file that creates an empty screen. In the event loop, 
print the event.key attribute whenever a pygame.KEYDOWN event is detected. Run 
the program and press various keys to see how Pygame responds. 


Shooting Bullets 


settings.py 


bullet.py 


Now let's add the ability to shoot bullets. We'll write code that fires a bullet, 
which is represented by a small rectangle, when the player presses the space- 
bar. Bullets will then travel straight up the screen until they disappear off 
the top of the screen. 


Adding the Bullet Settings 


At the end of the init () method, we'll update settings.py to include the 
values we'll need for a new Bullet class: 


# Bullet settings 

self.bullet speed - 2.0 
self.bullet width = 3 
self.bullet height - 15 
self.bullet color - (60, 60, 60) 


These settings create dark gray bullets with a width of 3 pixels and a 
height of 15 pixels. The bullets will travel slightly faster than the ship. 


Creating the Bullet Class 


Now create a bullet.py file to store our Bullet class. Here's the first part of 
bullet.py: 


import pygame 
from pygame.sprite import Sprite 


class Bullet(Sprite): 
"""A class to manage bullets fired from the ship.""" 
def init (self, ai game): 
"""Create a bullet object at the ship's current position. 
super(). init () 
self.screen - ai game.screen 
self.settings - ai game.settings 
self.color = self.settings.bullet color 


# Create a bullet rect at (0, 0) and then set correct position. 

self.rect = pygame.Rect(0, 0, self.settings.bullet width, 
self.settings.bullet height) 

self.rect.midtop - ai game.ship.rect.midtop 


# Store the bullet's position as a float. 
self.y - float(self.rect.y) 


The Bullet class inherits from Sprite, which we import from the pygame 
.sprite module. When you use sprites, you can group related elements in 
your game and act on all the grouped elements at once. To create a bullet 
instance, init ()needs the current instance of AlienInvasion, and we call 
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super() to inherit properly from Sprite. We also set attributes for the screen 
and settings objects, and for the bullet’s color. 

Next we create the bullet's rect attribute 6. The bullet isn’t based on an 
image, so we have to build a rect from scratch using the pygame.Rect() class. 
This class requires the x- and y-coordinates of the top-left corner of the 
rect, and the width and height of the rect. We initialize the rect at (0, 0), 
but we'll move it to the correct location in the next line, because the bullet's 
position depends on the ship's position. We get the width and height of the 
bullet from the values stored in self.settings. 

We set the bullet’s midtop attribute to match the ship's midtop attribute 6. 
This will make the bullet emerge from the top of the ship, making it look like 
the bullet is fired from the ship. We use a float for the bullet's y-coordinate so 
we can make fine adjustments to the bullet's speed ®. 

Here's the second part of bullet.py, update() and draw bullet(): 


def update(self): 
"""Move the bullet up the screen. 
# Update the exact position of the bullet. 
self.y -- self.settings.bullet speed 
# Update the rect position. 
self.rect.y = self.y 


def draw_bullet(self): 
"""Draw the bullet to the screen. 
pygame.draw.rect(self.screen, self.color, self.rect) 


The update() method manages the bullet’s position. When a bullet is 
fired, it moves up the screen, which corresponds to a decreasing y-coordinate 
value. To update the position, we subtract the amount stored in settings 
.bullet speed from self.y ®. We then use the value of self.y to set the value 
of self.rect.y O. 

The bullet speed setting allows us to increase the speed of the bullets 
as the game progresses or as needed to refine the game's behavior. Once a 
bullet is fired, we never change the value of its x-coordinate, so it will travel 
vertically in a straight line even if the ship moves. 

When we want to draw a bullet, we call draw bullet(). The draw.rect() 
function fills the part of the screen defined by the bullet's rect with the 
color stored in self.color 6. 


Storing Bullets in a Group 


Now that we have a Bullet class and the necessary settings defined, we can 
write code to fire a bullet each time the player presses the spacebar. We'll 
create a group in AlienInvasion to store all the active bullets so we can man- 
age the bullets that have already been fired. This group will be an instance 
of the pygame. sprite.Group class, which behaves like a list with some extra 
functionality that's helpful when building games. We'll use this group 
to draw bullets to the screen on each pass through the main loop and to 
update each bullet's position. 


First, we'll import the new Bullet class: 
alien_invasion.py 


from bullet import Bullet 


Next we'll create the group that holds the bullets in _init__(): 
alien_invasion.py 


self.bullets = pygame. sprite.Group() 


Then we need to update the position of the bullets on each pass through 
the while loop: 


alien_invasion.py 


self.bullets. update( ) 


When you call update() on a group, the group automatically calls 
update() for each sprite in the group. The line self.bullets.update() calls 
bullet.update() for each bullet we place in the group bullets. 


Firing Bullets 


In AlienInvasion, we need to modify _check_keydown_events() to fire a bullet 
when the player presses the spacebar. We don’t need to change _check_keyup 
_events() because nothing happens when the spacebar is released. We also 
need to modify _update_screen() to make sure each bullet is drawn to the 
screen before we call flip(). 

There will be a bit of work to do when we fire a bullet, so let’s write a 
new method, fire bullet(), to handle this work: 


alien 
_invasion.py 


e elif event.key -- pygame.K SPACE: 
self. fire bullet() 


def fire bullet(self): 
"""Create a new bullet and add it to the bullets group. 
new bullet - Bullet(self) 
self.bullets.add(new bullet) 


o0 
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for bullet in self.bullets.sprites(): 
bullet.draw bullet() 


We call fire bullet() when the spacebar is pressed €. In fire bullet(), 
we make an instance of Bullet and call it new bullet 6. We then add it to 
the group bullets using the add() method 9. The add() method is similar to 
append(), but it's written specifically for Pygame groups. 

The bullets.sprites() method returns a list of all sprites in the group 
bullets. To draw all fired bullets to the screen, we loop through the sprites 
in bullets and call draw bullet() on each one O. We place this loop before 
the line that draws the ship, so the bullets don't start out on top of the ship. 

When you run alien, invasion.py now, you should be able to move the ship 
right and left and fire as many bullets as you want. The bullets travel up the 
screen and disappear when they reach the top, as shown in Figure 12-3. You 
can alter the size, color, and speed of the bullets in settings.py. 


Figure 12-3: The ship after firing a series of bullets 


Deleting Old Bullets 


At the moment, the bullets disappear when they reach the top, but only 
because Pygame can't draw them above the top of the screen. The bullets 
actually continue to exist; their y-coordinate values just grow increasingly 
negative. This is a problem because they continue to consume memory and 
processing power. 


alien 
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settings.py 


We need to get rid of these old bullets, or the game will slow down from 
doing so much unnecessary work. To do this, we need to detect when the 
bottom value of a bullet’s rect has a value of 0, which indicates the bullet has 
passed off the top of the screen: 


# Get rid of bullets that have disappeared. 
for bullet in self.bullets.copy(): 
if bullet.rect.bottom <= 0: 
self.bullets.remove(bullet) 
print(len(self.bullets)) 


When you use a for loop with a list (or a group in Pygame), Python 
expects that the list will stay the same length as long as the loop is running. 
That means you can’t remove items from a list or group within a for loop, 
so we have to loop over a copy of the group. We use the copy() method to set 
up the for loop @, which leaves us free to modify the original bullets group 
inside the loop. We check each bullet to see whether it has disappeared off 
the top of the screen @. If it has, we remove it from bullets 6. We insert a 
print() call to show how many bullets currently exist in the game and verify 
they're being deleted when they reach the top of the screen O. 

If this code works correctly, we can watch the terminal output while fir- 
ing bullets and see that the number of bullets decreases to zero after each 
series of bullets has cleared the top of the screen. After you run the game 
and verify that bullets are being deleted properly, remove the print() call. If 
you leave it in, the game will slow down significantly because it takes more 
time to write output to the terminal than it does to draw graphics to the 
game window. 


Limiting the Number of Bullets 


Many shooting games limit the number of bullets a player can have on the 
screen at one time; doing so encourages players to shoot accurately. We'll 
do the same in Alien Invasion. 

First, store the number of bullets allowed in settings.py: 


self.bullets allowed = 3 7 
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This limits the player to three bullets at a time. We'll use this setting in 
AlienInvasion to check how many bullets exist before creating a new bullet 
in fire bullet(): 


if len(self.bullets) « self.settings.bullets allowed: 


When the player presses the spacebar, we check the length of bullets. 
If len(self.bullets) is less than three, we create a new bullet. But if three 
bullets are already active, nothing happens when the spacebar is pressed. 
When you run the game now, you should only be able to fire bullets in 
groups of three. 


Creating the update bullets() Method 


We want to keep the AlienInvasion class reasonably well organized, so now 
that we've written and checked the bullet management code, we can move 
it to a separate method. We'll create a new method called update bullets() 
and add it just before update screen(): 


def update bullets(self): 
"""Update position of bullets and get rid of old bullets.""" 
# Update bullet positions. 
,eltf.bu et L date() 


The code for _update_bullets() is cut and pasted from run_game(); all 
we've done here is clarify the comments. 
The while loop in run game() looks simple again: 


self . update bullets Q 


Now our main loop contains only minimal code, so we can quickly read 
the method names and understand what’s happening in the game. The 
main loop checks for player input, and then updates the position of the 
ship and any bullets that have been fired. We then use the updated posi- 
tions to draw a new screen and tick the clock at the end of each pass 
through the loop. 


Run alien_invasion.py one more time, and make sure you can still fire 
bullets without errors. 


TRY IT YOURSELF 


12-6. Sideways Shooter: Write a game that places a ship on the left side of the 


screen and allows the player to move the ship up and down. Make the ship fire 
a bullet that travels right across the screen when the player presses the space- 
bar. Make sure bullets are deleted once they disappear off the screen. 


Summary 


In this chapter, you learned to make a plan for a game and learned the 
basic structure of a game written in Pygame. You learned to set a back- 
ground color and store settings in a separate class where you can adjust 
them more easily. You saw how to draw an image to the screen and give the 
player control over the movement of game elements. You created elements 
that move on their own, like bullets flying up a screen, and you deleted 
objects that are no longer needed. You also learned to refactor code in a 
project on a regular basis to facilitate ongoing development. 

In Chapter 13, we’ll add aliens to Alien Invasion. By the end of the 
chapter, you'll be able to shoot down aliens, hopefully before they reach 
your ship! 
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In this chapter, we’ll add aliens to Alien 
Invasion. We'll add one alien near the top 
of the screen and then generate a whole fleet 

of aliens. We'll make the fleet advance sideways 
and down, and we'll get rid of any aliens hit by a bullet. 
Finally, we'll limit the number of ships a player has and 
end the game when the player runs out of ships. 


As you work through this chapter, you'll learn more about Pygame 
and about managing a large project. You'll also learn to detect collisions 
between game objects, like bullets and aliens. Detecting collisions helps you 
define interactions between elements in your games. For example, you can 
confine a character inside the walls of a maze or pass a ball between two 
characters. We'll continue to work from a plan that we revisit occasionally 
to maintain the focus of our code-writing sessions. 

Before we start writing new code to add a fleet of aliens to the screen, 
let's look at the project and update our plan. 
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Reviewing the Project 


When you're beginning a new phase of development on a large project, 
it’s always a good idea to revisit your plan and clarify what you want to 
accomplish with the code you're about to write. In this chapter, we'll do 
the following: 


Add a single alien to the top-left corner of the screen, with appropriate 
spacing around it. 


Fill the upper portion of the screen with as many aliens as we can fit 
horizontally. We'll then create additional rows of aliens until we have a 
full fleet. 


Make the fleet move sideways and down until the entire fleet is shot 
down, an alien hits the ship, or an alien reaches the ground. If the 
entire fleet is shot down, we'll create a new fleet. If an alien hits the 
ship or the ground, we'll destroy the ship and create a new fleet. 
Limit the number of ships the player can use, and end the game when 
the player has used up the allotted number of ships. 


We'll refine this plan as we implement features, but this is specific 


enough to start writing code. 


You should also review your existing code when you begin working on a 


new series of features in a project. Because each new phase typically makes 
a project more complex, it's best to clean up any cluttered or inefficient 
code. We've been refactoring as we go, so there isn't any code that we need 
to refactor at this point. 


Creating the First Alien 


Chapter 13 


Placing one alien on the screen is like placing a ship on the screen. Each 


alien's behavior is controlled by a class called Alien, which we'll structure 
like the Ship class. We'll continue using bitmap images for simplicity. You 


can find your own image for an alien or use the one shown in Figure 13-1, 


which is available in the book's resources at hitps://ehmatthes.github.io/pcc_3e. 
This image has a gray background, which matches the screen's background 
color. Make sure you save the image file you choose in the images folder. 
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Figure 13-1: The alien we'll use 


to build the fleet 


alien.py 


alien_invasion.py 


Creating the Alien Class 


Now we'll write the Alien class and save it as alien.py: 


import pygame 
from pygame.sprite import Sprite 


class Alien(Sprite): 
"""A class to represent a single alien in the fleet.""" 
def init (self, ai game): 
"""Initialize the alien and set its starting position. 
super(). init () 
self.screen - ai game.screen 


# Load the alien image and set its rect attribute. 
self.image - pygame.image.load('images/alien.bmp') 
self.rect - self.image.get rect() 


# Start each new alien near the top left of the screen. 
self.rect.x = self.rect.width 
self.rect.y = self.rect.height 


# Store the alien's exact horizontal position. 
self.x = float(self.rect.x) 


Most of this class is like the Ship class, except for the alien’s placement on 
the screen. We initially place each alien near the top-left corner of the screen; 
we add a space to the left of it that’s equal to the alien’s width and a space 
above it equal to its height 9, so it's easy to see. We're mainly concerned with 
the aliens’ horizontal speed, so we'll track the horizontal position of each 
alien precisely 8. 

This Alien class doesn't need a method for drawing it to the screen; 
instead, we'll use a Pygame group method that automatically draws all the 
elements of a group to the screen. 


Creating an Instance of the Alien 


We want to create an instance of Alien so we can see the first alien on the 
screen. Because it's part of our setup work, we'll add the code for this 
instance at the end of the init () method in AlienInvasion. Eventually, 
we'll create an entire fleet of aliens, which will be quite a bit of work, so we'll 
make a new helper method called create fleet(). 

The order of methods in a class doesn't matter, as long as there's some 
consistency to how they're placed. I'll place create fleet() just before the 
update screen() method, but anywhere in AlienInvasion will work. First, we'll 
import the Alien class. 

Here are the updated import statements for alien invasion.py. 


from alien import Alien 
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And here's the updated init () method: 


alien invasion.py 


el 
self.aliens - pygame.sprite.Group() 


self. create fleet() 


We create a group to hold the fleet of aliens, and we call create fleet(), 
which we're about to write. 
Here's the new create fleet() method: 


alien invasion.py def create fleet(self): 
"""Create the fleet of aliens. 
# Make an alien. 
alien - Alien(self) 
self.aliens.add(alien) 


In this method, we're creating one instance of Alien and then adding it 
to the group that will hold the fleet. T'he alien will be placed in the default 
upper-left area of the screen. 

To make the alien appear, we need to call the group's draw() method in 
update screen(): 


alien invasion.py 


self. aliens. draw(self. screen) 


When you call draw() on a group, Pygame draws each element in the 
group at the position defined by its rect attribute. The draw() method 
requires one argument: a surface on which to draw the elements from the 
group. Figure 13-2 shows the first alien on the screen. 


PR 


Figure 13-2: The first alien appears. 
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Now that the first alien appears correctly, we'll write the code to draw 


an entire fleet. 


Building the Alien Fleet 


To draw a fleet, we need to figure out how to fill the upper portion of the 
screen with aliens, without overcrowding the game window. There are a num- 
ber of ways to accomplish this goal. We'll approach it by adding aliens across 
the top of the screen, until there's no space left for a new alien. Then we'll 
repeat this process, as long as we have enough vertical space to add a new row. 


alien 
_invasion.py 
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Creating a Row of Aliens 


Now we're ready to generate a full row of aliens. To make a full row, we'll 
first make a single alien so we have access to the alien's width. We'll place 
an alien on the left side of the screen and then keep adding aliens until we 


run out of space: 


# Create an alien and keep adding aliens until there's no room left. 


# Spacing between aliens is one alien width. 
alien width - alien.rect.width 


current x - alien width 


while current x « (self.settings.screen width - 2 * alien width): 


new alien - Alien(self) 

new alien.x = current x 

new alien.rect.x - current x 
self.aliens.add(new alien) 
current x *- 2 * alien width 


We get the alien's width from the first alien we created, and then define 


a variable called current x ®. This refers to the horizontal position of the 


next alien we intend to place on the screen. We initially set this to one alien 
width, to offset the first alien in the fleet from the left edge of the screen. 


Next, we begin the while loop @; we're going to keep adding aliens 


while there's enough room to place one. To determine whether there's room 
to place another alien, we'll compare current x to some maximum value. A 


first attempt at defining this loop might look like this: 


while current x < self.settings.screen width: 


This seems like it might work, but it would place the final alien in the 


row at the far-right edge of the screen. So we add a little margin on the 
right side of the screen. As long as there's at least two alien widths' worth 


of space at the right edge of the screen, we enter the loop and add another 


alien to the fleet. 
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Whenever there’s enough horizontal space to continue the loop, we 
want to do two things: create an alien at the correct position, and define 
the horizontal position of the next alien in the row. We create an alien and 
assign it to new alien ©. Then we set the precise horizontal position to the 
current value of current x O. We also position the alien's rect at this same 
x-value, and add the new alien to the group self.aliens. 

Finally, we increment the value of current x 6. We add two alien widths 
to the horizontal position, to move past the alien we just added and to leave 
some space between the aliens as well. Python will re-evaluate the condition 
at the start of the while loop and decide if there's room for another alien. 
When there's no room left, the loop will end, and we should have a full row 
of aliens. 

When you run Alien Invasion now, you should see the first row of aliens 
appear, as in Figure 13-3. 
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Figure 13-3: The first row of aliens 


It’s not always obvious exactly how to construct a loop like the one shown in this sec- 
tion. One nice thing about programming is that your initial ideas for how to approach 
a problem like this don't have to be correct. It’s perfectly reasonable to start a loop like 
this with the aliens positioned too far to the right, and then modify the loop until you 
have an appropriate amount of space on the screen. 


Refactoring create fleet() 


If the code we've written so far was all we needed to create a fleet, we'd 
probably leave create fleet() as is. But we have more work to do, so let's 
clean up the method a bit. We'll add a new helper method, create alien(), 
and callitfrom create fleet(): 


e current x « (self.settings.screen width 
self. create alien(current x) 


current x += 2 * alien width 


e 
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def create alien(self, x position): 

"""Create an alien and place it in the row.""" 
new alien.x = x position 
new alien.rect.x = x position 


The method create alien() requires one parameter in addition to self: 
the x-value that specifies where the alien should be placed 9. The code 
in the body of create alien() is the same code that was in. create fleet(), 
except we use the parameter name x position in place of current x. This 
refactoring will make it easier to add new rows and create an entire fleet. 


Adding Rows 


To finish the fleet, we'll keep adding more rows until we run out of room. 
We'll use a nested loop—we'll wrap another while loop around the current 
one. The inner loop will place aliens horizontally in a row by focusing on 
the aliens’ x-values. The outer loop will place aliens vertically by focusing 
on the y-values. We'll stop adding rows when we get near the bottom of the 
screen, leaving enough space for the ship and some room to start firing at 
the aliens. 

Here's how to nest the two while loops in create fleet(): 


Create an iien and Keep adding alle til there's no roon le: 1 
# Spacing between aliens is one alien width and one alien height. 


alien width, alien height - alien.rect.size 
current x, current y - alien width, alien height 
while current y « (self.settings.screen height - 3 * alien height): 
self. create alien(current x, current y) 
# Finished a row; reset x value, and increment y value. 


current x - alien width 
current y += 2 * alien height 


We'll need to know the alien's height in order to place rows, so we grab 
the alien’s width and height using the size attribute of an alien rect 9. A 
rect's size attribute is a tuple containing its width and height. 

Next, we set the initial x- and y-values for the placement of the first alien 
in the fleet @. We place it one alien width in from the left and one alien 
height down from the top. Then we define the while loop that controls how 
many rows are placed onto the screen ®. As long as the y-value for the next 
row is less than the screen height, minus three alien heights, we'll keep 
adding rows. (If this doesn't leave the right amount of space, we can 
adjust it later.) 


Aliens! 261 


We call create alien(), and pass it the y-value as well as its x-position O. 
We'll modify create alien() in a moment. 

Notice the indentation of the last two lines of code 9. They're inside the 
outer while loop, but outside the inner while loop. This block runs after the 
inner loop is finished; it runs once after each row is created. After each 
row has been added, we reset the value of current x so the first alien in the 
next row will be placed at the same position as the first alien in the previous 
rows. Then we add two alien heights to the current value of current y, so the 
next row will be placed further down the screen. Indentation is really impor- 
tant here; if you don’t see the correct fleet when you run alien invasion.py 
at the end of this section, check the indentation of all the lines in these 
nested loops. 

We need to modify create alien() to set the vertical position of the 
alien correctly: 


def create alien(self, x position, y position): 
"""Create an alien and place it in the fleet. 
new alien - Alien(self) 
new alien.x = x position 
new alien.rect.x = x position 
new alien.rect.y - y position 
self.aliens.add(new alien) 


We modify the definition of the method to accept the y-value for the 
new alien, and we set the vertical position of the rect in the body of the 
method. 

When you run the game now, you should see a full fleet of aliens, as 
shown in Figure 13-4. 
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Figure 13-4: The full fleet appears. 


In the next section, we'll make the fleet move! 
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TRY IT YOURSELF 


13-1. Stars: Find an image of a star. Make a grid of stars appear on the screen. 


13-2. Better Stars: You can make a more realistic star pattern by introducing 


randomness when you place each star. Recall from Chapter 9 that you can get 


a random number like this: 


from random import randint 
random number = randint(-10, 10) 


This code returns a random integer between -10 and 10. Using your code 
in Exercise 13-1, adjust each star's position by a random amount. 


Making the Fleet Move 
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Moving the Aliens Right 


To move the aliens, we'll use an update() method in alien.py, which we'll 
call for each alien in the group of aliens. First, add a setting to control the 
speed of each alien: 


Now let's make the fleet of aliens move to the right across the screen until 
it hits the edge, and then make it drop a set amount and move in the other 
direction. We'll continue this movement until all aliens have been shot 
down, one collides with the ship, or one reaches the bottom of the screen. 
Let's begin by making the fleet move to the right. 


# Alien settings 
self.alien speed - 1.0 


Then use this setting to implement update() in alien.py: 


self.settings - ai game.settings 


def update(self): 
"""Move the alien to the right. 
self.x += self.settings.alien speed 
self.rect.x = self.x 


We create a settings parameterin init ()so we can access the alien's 
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speed in update(). Each time we update an alien's position, we move it to the 
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right by the amount stored in alien speed. We track the alien's exact position 
with the self.x attribute, which can hold float values 9. We then use the 
value of self.x to update the position of the alien’s rect 6. 

In the main while loop, we already have calls to update the ship and bul- 
let positions. Now we'll add a call to update the position of each alien as well: 


self. update aliens() 


We're about to write some code to manage the movement of the fleet, 
so we create a new method called update aliens(). We update the aliens' 
positions after the bullets have been updated, because we'll soon be check- 
ing to see whether any bullets hit any aliens. 

Where you place this method in the module is not critical. But to keep the 
code organized, I'll place itjust after update bullets() to match the order of 
method calls in the while loop. Here's the first version of. update aliens(): 


def update aliens(self): 
"""Update the positions of all aliens in the fleet. 
self.aliens.update() 


We use the update() method on the aliens group, which calls each 
alien's update() method. When you run Alien Invasion now, you should see 
the fleet move right and disappear off the side of the screen. 


Creating Settings for Fleet Direction 


Now we'll create the settings that will make the fleet move down the screen 
and to the left when it hits the right edge of the screen. Here's how to 
implement this behavior: 


self.alien speed = 
self.fleet_drop speed = 10 

# fleet_direction of 1 represents right; -1 represents left. 
self.fleet_direction = 1 


The setting fleet_drop_speed controls how quickly the fleet drops down 
the screen each time an alien reaches either edge. It’s helpful to separate 
this speed from the aliens’ horizontal speed so you can adjust the two 
speeds independently. 

To implement the setting fleet_direction, we could use a text value such 
as 'left' or 'right', but we'd end up with if-elif statements testing for the 
fleet direction. Instead, because we only have two directions to deal with, 
let's use the values 1 and -1 and switch between them each time the fleet 
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changes direction. (Using numbers also makes sense because moving right 
involves adding to each alien's x-coordinate value, and moving left involves 
subtracting from each alien's x-coordinate value.) 


Checking Whether an Alien Has Hit the Edge 


We need a method to check whether an alien is at either edge, and we need 
to modify update() to allow each alien to move in the appropriate direction. 
This code is part of the Alien class: 


def check edges(self): 
"""Return True if alien is at edge of screen. 
screen rect - self.screen.get rect() 
return (self.rect.right >= screen rect.right) or (self.rect.left <= 0) 


Move the alien right or left. 
self.x += self.settings.alien speed * self.settings.fleet direction 


We can call the new method check edges() on any alien to see whether 
it's at the left or right edge. The alien is at the right edge if the right attri- 
bute of its rect is greater than or equal to the right attribute of the screen's 
rect. It's at the left edge if its left value is less than or equal to 0 6. Rather 
than put this conditional test in an if block, we put the test directly in the 
return statement. This method will return True if the alien is at the right or 
left edge, and False if it is not at either edge. 

We modify the method update() to allow motion to the left or right 
by multiplying the alien's speed by the value of fleet direction @. If fleet 
. direction is 1, the value of alien speed will be added to the alien's current 
position, moving the alien to the right; if fleet directionis -l, the value will 
be subtracted from the alien's position, moving the alien to the left. 


Dropping the Fleet and Changing Direction 


When an alien reaches the edge, the entire fleet needs to drop down and 
change direction. Therefore, we need to add some code to AlienInvasion 
because that's where we'll check whether any aliens are at the left or right 
edge. We'll make this happen by writing the methods check fleet edges() 
and change fleet direction(), and then modifying update aliens(). I'll put 
these new methods after create alien(), but again, the placement of these 
methods in the class isn't critical. 


def check fleet edges(self): 
"""Respond appropriately if any aliens have reached an edge. 
for alien in self.aliens.sprites(): 
if alien.check edges(): 
self. change fleet direction() 
break 


def change fleet direction(self): 
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Drop the entire fleet and change the fleet's direction. 
for alien in self.aliens.sprites(): 

e alien.rect.y += self.settings.fleet drop speed 
self.settings.fleet direction *- -1 


In check fleet edges(), we loop through the fleet and call check edges() on 
each alien @. If check edges() returns True, we know an alien is at an edge and 
the whole fleet needs to change direction; so we call change fleet direction() 
and break out of the loop 0. In change fleet direction(), we loop through 
all the aliens and drop each one using the setting fleet drop speed ©; then 
we change the value of fleet direction by multiplying its current value by -1. 
The line that changes the fleet's direction isn't part of the for loop. We want to 
change each alien's vertical position, but we only want to change the direction 
of the fleet once. 

Here are the changes to. update aliens(): 
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"""Check if the fleet is at an edge, then update positions. 
self. check fleet edges() 


We've modified the method by calling check fleet edges() before 
updating each alien's position. 

When you run the game now, the fleet should move back and forth 
between the edges of the screen and drop down every time it hits an edge. 
Now we can start shooting down aliens and watch for any aliens that hit the 
ship or reach the bottom of the screen. 


TRY IT YOURSELF 


13-3. Raindrops: Find an image of a raindrop and create a grid of raindrops. 
Make the raindrops fall toward the bottom of the screen until they disappear. 


13-4. Steady Rain: Modify your code in Exercise 13-3 so when a row of rain- 


drops disappears off the bottom of the screen, a new row appears at the top of 
the screen and begins to fall. 


Shooting Aliens 


We've built our ship and a fleet of aliens, but when the bullets reach the 
aliens, they simply pass through because we aren't checking for collisions. In 
game programming, collisions happen when game elements overlap. To make 
the bullets shoot down aliens, we'll use the function sprite.groupcollide() to 
look for collisions between members of two groups. 
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Detecting Bullet Collisions 


We want to know right away when a bullet hits an alien so we can make an 
alien disappear as soon as it’s hit. To do this, we’ll look for collisions imme- 
diately after updating the position of all the bullets. 

The sprite.groupcollide() function compares the rects of each element in 
one group with the rects of each element in another group. In this case, it com- 
pares each bullet’s rect with each alien’s rect and returns a dictionary contain- 
ing the bullets and aliens that have collided. Each key in the dictionary will be 
a bullet, and the corresponding value will be the alien that was hit. (We'll also 
use this dictionary when we implement a scoring system in Chapter 14.) 

Add the following code to the end of update bullets() to check for colli- 
sions between bullets and aliens: 


# Check for any bullets that have hit aliens. 

# If so, get rid of the bullet and the alien. 

collisions - pygame.sprite.groupcollide( 
self.bullets, self.aliens, True, True) 


The new code we added compares the positions of all the bullets in 
self.bullets and all the aliens in self.aliens, and identifies any that overlap. 
Whenever the rects of a bullet and alien overlap, groupcollide() adds a key- 
value pair to the dictionary it returns. The two True arguments tell Pygame 
to delete the bullets and aliens that have collided. (To make a high-powered 
bullet that can travel to the top of the screen, destroying every alien in its 
path, you could set the first Boolean argument to False and keep the second 
Boolean argument set to True. The aliens hit would disappear, but all bullets 
would stay active until they disappeared off the top of the screen.) 

When you run Alien Invasion now, aliens you hit should disappear. 
Figure 13-5 shows a fleet that has been partially shot down. 
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Figure 13-5: We can shoot aliens! 
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Making Larger Bullets for Testing 


You can test many features of Alien Invasion simply by running the game, 
but some features are tedious to test in the normal version of the game. For 
example, it’s a lot of work to shoot down every alien on the screen multiple 
times to test whether your code responds to an empty fleet correctly. 

To test particular features, you can change certain game settings to 
focus on a particular area. For example, you might shrink the screen so 
there are fewer aliens to shoot down or increase the bullet speed and give 
yourself lots of bullets at once. 

My favorite change for testing Alien Invasion is to use really wide bul- 
lets that remain active even after they’ve hit an alien (see Figure 13-6). Try 
setting bullet_width to 300, or even 3,000, to see how quickly you can shoot 
down the fleet! 
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Figure 13-6: Extra-powerful bullets make some aspects of the game 
easier to test. 


Changes like these will help you test the game more efficiently and pos- 
sibly spark ideas for giving players bonus powers. Just remember to restore 
the settings to normal when you're finished testing a feature. 


Repopulating the Fleet 


One key feature of Alien Invasion is that the aliens are relentless: every time 
the fleet is destroyed, a new fleet should appear. 

To make a new fleet of aliens appear after a fleet has been destroyed, 
we first check whether the aliens group is empty. If it is, we make a call 
to create fleet(). We'll perform this check at the end of _update_bullets(), 
because that's where individual aliens are destroyed. 


def update bullets(self): 
--snip-- 


if not self.aliens: 
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# Destroy existing bullets and create new fleet. 
self.bullets.empty() 
self. create fleet() 


We check whether the aliens group is empty 6. An empty group evaluates 
to False, so this is a simple way to check whether the group is empty. If it is, we 
get rid of any existing bullets by using the empty() method, which removes 
all the remaining sprites from a group 9. We also call create fleet(), which 
fills the screen with aliens again. 

Now a new fleet appears as soon as you destroy the current fleet. 


Speeding Up the Bullets 


If you've tried firing at the aliens in the game's current state, you might find 
that the bullets aren't traveling at the best speed for gameplay. They might 
be a little too slow or a little too fast. At this point, you can modify the set- 
tings to make the gameplay more interesting. Keep in mind that the game 
is going to get progressively faster, so don't make the game too fast at the 
beginning. 

We modify the speed of the bullets by adjusting the value of bullet speed 
in settings.py. On my system, I'll adjust the value of bullet speed to 2.5, so the 
bullets travel a little faster: 


self.bullet speed - 2.5 


The best value for this setting depends on your experience of the game, 
so find a value that works for you. You can adjust other settings as well. 


Refactoring update bullets() 


Let's refactor update bullets() so it's not doing so many different tasks. 
We'll move the code for dealing with bullet-alien collisions to a separate 
method: 


self. check bullet alien collisions() 


def check bullet alien collisions(self): 
"""Respond to bullet-alien collisions. 
# Remove any bullets and aliens that have collided. 


el 
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We've created a new method, check bullet alien collisions(), to look 
for collisions between bullets and aliens and to respond appropriately if the 
entire fleet has been destroyed. Doing so keeps update bullets() from grow- 
ing too long and simplifies further development. 


TRY IT YOURSELF 


13-5. Sideways Shooter Part 2: We've come a long way since Exercise 12-6, 
Sideways Shooter. For this exercise, try to develop Sideways Shooter to the 


same point we've brought Alien Invasion to. Add a fleet of aliens, and make 
them move sideways toward the ship. Or, write code that places aliens at ran- 
dom positions along the right side of the screen and then sends them toward 
the ship. Also, write code that makes the aliens disappear when they're hit. 


Ending the Game 


What's the fun and challenge in playing a game you can't lose? If the player 
doesn't shoot down the fleet quickly enough, we'll have the aliens destroy 
the ship when they make contact. At the same time, we'll limit the number 
of ships a player can use, and we'll destroy the ship when an alien reaches 
the bottom of the screen. The game will end when the player has used up 
all their ships. 


Detecting Alien-Ship Collisions 

We'll start by checking for collisions between aliens and the ship so we can 
respond appropriately when an alien hits it. We'll check for alien-ship colli- 
sions immediately after updating the position of each alien in AlienInvasion: 


alien 
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# Look for alien-ship collisions. 
e if pygame.sprite.spritecollideany(self.ship, self.aliens): 
e print("Ship hit!!!") 


The spritecollideany() function takes two arguments: a sprite and a 
group. The function looks for any member of the group that has collided 
with the sprite and stops looping through the group as soon as it finds one 
member that has collided with the sprite. Here, it loops through the group 
aliens and returns the first alien it finds that has collided with ship. 
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If no collisions occur, spritecollideany() returns None and the if block 
won't execute 8. If it finds an alien that has collided with the ship, it returns 
that alien and the if block executes: it prints Ship hit!!! 6. When an alien 
hits the ship, we'll need to do a number of tasks: delete all remaining aliens 
and bullets, recenter the ship, and create a new fleet. Before we write code 
to do all this, we want to know that our approach to detecting alien-ship col- 
lisions works correctly. Writing a print() call is a simple way to ensure we're 
detecting these collisions properly. 

Now when you run Alien Invasion, the message Ship hit!!! should appear 
in the terminal whenever an alien runs into the ship. When you're testing 
this feature, set fleet drop speed to a higher value, such as 50 or 100, so the 
aliens reach your ship faster. 


Responding to Alien-Ship Collisions 


Now we need to figure out exactly what will happen when an alien collides 
with the ship. Instead of destroying the ship instance and creating a new 
one, we'll count how many times the ship has been hit by tracking statistics 
for the game. Tracking statistics will also be useful for scoring. 

Let's write a new class, GameStats, to track game statistics, and let's save it 
as game stats.py: 


class GameStats: 
"""Track statistics for Alien Invasion.""" 
def init (self, ai game): 
"""Initialize statistics. 
self.settings - ai game.settings 
self.reset stats() 


def reset stats(self): 
"""Initialize statistics that can change during the game. 
self.ships left = self.settings.ship limit 


We'll make one GameStats instance for the entire time Alien Invasion is 
running, but we'll need to reset some statistics each time the player starts a 
new game. To do this, we'll initialize most of the statistics in the reset stats() 
method, instead of directly in init (). We'll call this method from — init () 
so the statistics are set properly when the GameStats instance is first created 6. 
But we'll also be able to call reset stats() anytime the player starts a new game. 
Right now we have only one statistic, ships left, the value of which will change 
throughout the game. 

The number of ships the player starts with should be stored in settings.py 
as ship limit: 


self.ship limit - 3 
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We also need to make a few changes in alien_invasion.py to create an 
instance of GameStats. First, we'll update the import statements at the top of 
the file: 


from time import sleep 


from game stats import GameStats 


We import the sleep() function from the time module in the Python 
standard library, so we can pause the game for a moment when the ship is 
hit. We also import GameStats. 

We'll create an instance of GameStats in init (): 


# Create an instance to store game statistics. 
self.stats - GameStats(self) 


We make the instance after creating the game window but before defin- 
ing other game elements, such as the ship. 

When an alien hits the ship, we'll subtract 1 from the number of ships 
left, destroy all existing aliens and bullets, create a new fleet, and reposition 
the ship in the middle of the screen. We'll also pause the game for a moment 
so the player can notice the collision and regroup before a new fleet appears. 

Let's put most of this code in a new method called ship hit(). We'll call 
this method from update aliens() when an alien hits the ship: 


def ship hit(self): 
"""Respond to the ship being hit by an alien. 
# Decrement ships left. 
self.stats.ships left -= 1 


# Get rid of any remaining bullets and aliens. 
self.bullets.empty() 
self.aliens.empty() 


# Create a new fleet and center the ship. 
self. create fleet() 
self.ship.center ship() 
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# Pause. 
sleep(0.5) 


The new method _ship_hit() coordinates the response when an alien 
hits a ship. Inside _ship_hit(), the number of ships left is reduced by 1 @, 
after which we empty the groups bullets and aliens @. 

Next, we create a new fleet and center the ship 9. (We'll add the 
method center ship() to Ship in a moment.) Then we add a pause after the 
updates have been made to all the game elements but before any changes 
have been drawn to the screen, so the player can see that their ship has 
been hit O. The sleep() call pauses program execution for half a second, 
long enough for the player to see that the alien has hit the ship. When the 
sleep() function ends, code execution moves on to the update screen() 
method, which draws the new fleet to the screen. 

In update aliens(), we replace the print() call with a call to. ship hit() 
when an alien hits the ship: 


self. ship hit() 


Here's the new method center ship(), which belongs in ship.py: 


def center ship(self): 
"""Center the ship on the screen. 
self.rect.midbottom - self.screen rect.midbottom 
self.x - float(self.rect.x) 


We center the ship the same way we did in init (). After centering it, 
we reset the self.x attribute, which allows us to track the ship's exact position. 


Notice that we never make more than one ship; we make only one ship instance for the 
whole game and recenter it whenever the ship has been hit. The statistic ships left 
will tell us when the player has run out of ships. 


Run the game, shoot a few aliens, and let an alien hit the ship. The game 
should pause, and a new fleet should appear with the ship centered at the 
bottom of the screen again. 


Aliens That Reach the Bottom of the Screen 


If an alien reaches the bottom of the screen, we'll have the game respond 
the same way it does when an alien hits the ship. To check when this hap- 
pens, add a new method in alien, invasion.py: 


def check aliens bottom(self): 
"""Check if any aliens have reached the bottom of the screen.""" 
for alien in self.aliens.sprites(): 
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e if alien.rect.bottom »- self.settings.screen height: 
# Treat this the same as if the ship got hit. 
self. ship hit() 
break 


The method check aliens bottom() checks whether any aliens have reached 
the bottom of the screen. An alien reaches the bottom when its rect.bottom 
value is greater than or equal to the screen's height 6. If an alien reaches the 
bottom, we call ship hit(). If one alien hits the bottom, there's no need to 
check the rest, so we break out of the loop after calling ship hit(). 

We'll call this method from update aliens(): 


alien invasion.py 


# Look for aliens hitting the bottom of the screen. 
self. check aliens bottom() 


We call check aliens bottom() after updating the positions of all the 
aliens and after looking for alien-ship collisions. Now a new fleet will 
appear every time the ship is hit by an alien or an alien reaches the bottom 
of the screen. 


Game Over! 


Alien Invasion feels more complete now, but the game never ends. The value 
of ships left just grows increasingly negative. Let's add a game active flag, so 
we can end the game when the player runs out of ships. We'll set this flag at 
theend ofthe init () method in AlienInvasion: 
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# Start Alien Invasion in an active state. 
self.game active - True 


Now we add code to. ship hit() that sets game active to False when the 
player has used up all their ships: 


alien invasion.py 
if self.stats.ships left > 0: 


else: 
self.game active - False 
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Most of ship hit() is unchanged. We've moved all the existing code 
into an if block, which tests to make sure the player has at least one ship 
remaining. If so, we create a new fleet, pause, and move on. If the player has 
no ships left, we set game active to False. 


Identifying When Parts of the Game Should Run 


We need to identify the parts of the game that should always run and the 
parts that should run only when the game is active: 
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if self.game active: 


In the main loop, we always need to call check events(), even if the 
game is inactive. For example, we still need to know if the user presses Q to 
quit the game or clicks the button to close the window. We also continue 
updating the screen so we can make changes to the screen while waiting to 
see whether the player chooses to start a new game. The rest of the function 
calls need to happen only when the game is active, because when the game 
is inactive, we don't need to update the positions of game elements. 

Now when you play Alien Invasion, the game should freeze when you've 
used up all your ships. 


TRY IT YOURSELF 


13-6. Game Over: In Sideways Shooter, keep track of the number of times the 


ship is hit and the number of times an alien is hit by the ship. Decide on an 


appropriate condition for ending the game, and stop the game when this situa- 
tion occurs. 


Summary 


In this chapter, you learned how to add a large number of identical elements 
to a game by creating a fleet of aliens. You used nested loops to create a 
grid of elements, and you made a large set of game elements move by call- 
ing each element’s update() method. You learned to control the direction of 


Aliens! 275 


objects on the screen and to respond to specific situations, such as when the 
fleet reaches the edge of the screen. You detected and responded to colli- 
sions when bullets hit aliens and aliens hit the ship. You also learned how 
to track statistics in a game and use a game_active flag to determine when 
the game is over. 

In the next and final chapter of this project, we'll add a Play button so 
the player can choose when to start their first game and whether to play 
again when the game ends. We'll speed up the game each time the player 
shoots down the entire fleet, and we'll add a scoring system. The final result 
will be a fully playable game! 
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SCORING 


In this chapter, we'll finish building Alzen 

Invasion. We'll add a Play button to start the 
game on demand and to restart the game 

once it ends. We’ll also change the game so it 

speeds up when the player moves up a level, and we’ll 
implement a scoring system. By the end of the chap- 
ter, you'll know enough to start writing games that 
increase in difficulty as a player progresses and that 
feature complete scoring systems. 
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Adding the Play Button 


button.py 


Chapter 14 


In this section, we’ll add a Play button that appears before a game begins 
and reappears when the game ends so the player can play again. 

Right now, the game begins as soon as you run alien_invasion.py. Let’s 
start the game in an inactive state and then prompt the player to click a Play 
button to begin. To do this, modify the init () method of AlienInvasion: 


# Start Alien Invasion in an inactive state. 
self.game active - False 


Now the game should start in an inactive state, with no way for the 
player to start it until we make a Play button. 


Creating a Button Class 


Because Pygame doesn't have a builtin method for making buttons, we'll 
write a Button class to create a filled rectangle with a label. You can use this 
code to make any button in a game. Here's the first part of the Button class; 
save it as button.py: 


import pygame.font 


class Button: 
"""A class to build buttons for the game.""" 
def init (self, ai game, msg): 
"""Initialize button attributes. 
self.screen - ai game.screen 
self.screen rect - self.screen.get rect() 


# Set the dimensions and properties of the button. 
self.width, self.height - 200, 50 

self.button color - (0, 135, O) 

self.text color - (255, 255, 255) 

self.font - pygame.font.SysFont(None, 48) 


# Build the button's rect object and center it. 
self.rect = pygame.Rect(0, 0, self.width, self.height) 
self.rect.center - self.screen rect.center 


# The button message needs to be prepped only once. 
self. prep msg(msg) 


First, we import the pygame. font module, which lets Pygame render text 
to the screen. The init () method takes the parameters self, the ai game 
object, and msg, which contains the button's text €. We set the button 
dimensions @, set button color to color the button’s rect object dark green, 
and set text color to render the text in white. 
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Next, we prepare a font attribute for rendering text ©. The None argu- 
ment tells Pygame to use the default font, and 48 specifies the size of the 
text. To center the button on the screen, we create a rect for the button O 
and set its center attribute to match that of the screen. 

Pygame works with text by rendering the string you want to display as 
an image. Finally, we call prep msg() to handle this rendering 6. 

Here's the code for prep msg(): 


def prep msg(self, msg): 
"""Turn msg into a rendered image and center text on the button. 
self.msg image - self.font.render(msg, True, self.text color, 
self.button color) 
self.msg image rect - self.msg image.get rect() 
self.msg image rect.center - self.rect.center 


The prep msg() method needs a self parameter and the text to be ren- 
dered as an image (msg). The call to font.render() turns the text stored in 
msg into an image, which we then store in self.msg image €. The font.render() 
method also takes a Boolean value to turn antialiasing on or off (antialiasing 
makes the edges of the text smoother). The remaining arguments are the 
specified font color and background color. We set antialiasing to True and set 
the text background to the same color as the button. (If you don't include 
a background color, Pygame will try to render the font with a transparent 
background.) 

We center the text image on the button by creating a rect from the 
image and setting its center attribute to match that of the button 6. 

Finally, we create a draw button() method that we can call to display the 
button onscreen: 


def draw button(self): 
"""Draw blank button and then draw message. 
self.screen.fill(self.button color, self.rect) 
self.screen.blit(self.msg image, self.msg image rect) 


We call screen. fill() to draw the rectangular portion of the button. Then 
we call screen.blit() to draw the text image to the screen, passing it an image 
and the rect object associated with the image. This completes the Button class. 


Drawing the Button to the Screen 


We'll use the Button class to create a Play button in AlienInvasion. First, we'll 
update the import statements: 


from button import Button 
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Because we need only one Play button, we’ll create the button in the 
. init () method of AlienInvasion. We can place this code at the very end 


of init (): 
alien invasion.py def init (self): 
--snip-- 


self.game active - False 


# Make the Play button. 
self.play button - Button(self, "Play") 


This code creates an instance of Button with the label Play, but it doesn't 
draw the button to the screen. To do this, we'll call the button's draw button() 
method in update screen(): 


alien invasion.py def update screen(self): 
--snip-- 
self.aliens.draw(self.screen) 
# Draw the play button if the game is inactive. 
if not self.game active: 


self.play button.draw button() 


pygame.display.flip() 


To make the Play button visible above all other elements on the screen, 
we draw it after all the other elements have been drawn but before flipping 
to a new screen. We include it in an if block, so the button only appears 
when the game is inactive. 

Now when you run Alien Invasion, you should see a Play button in the 
center of the screen, as shown in Figure 14-1. 


eo Alsen invasion 
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Figure 14-1: A Play button appears when the game is inactive. 
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Starting the Game 


To start a new game when the player clicks Play, add the following elif 
block to the end of check events() to monitor mouse events over the button: 


elif event.type == pygame.MOUSEBUTTONDOWN : 
mouse pos = pygame.mouse.get_pos() 
self. check play button(mouse pos) 


Pygame detects a MOUSEBUTTONDOWN event when the player clicks anywhere 
on the screen 9, but we want to restrict our game to respond to mouse clicks 
only on the Play button. To accomplish this, we use pygame.mouse.get pos(), 
which returns a tuple containing the mouse cursor's x- and y-coordinates 
when the mouse button is clicked @. We send these values to the new method 
check play button() ©. 

Here's check play button(), which I chose to place after check events(): 


def check play button(self, mouse pos): 
"""Start a new game when the player clicks Play. 
if self.play button.rect.collidepoint(mouse pos): 
self.game active - True 


We use the rect method collidepoint() to check whether the point of 
the mouse click overlaps the region defined by the Play button’s rect 6. If 
SO, we set game active to True, and the game begins! 

At this point, you should be able to start and play a full game. When the 
game ends, the value of game active should become False and the Play but- 
ton should reappear. 


Resetting the Game 


The Play button code we just wrote works the first time the player clicks 
Play. But it doesn't work after the first game ends, because the conditions 
that caused the game to end haven't been reset. 

To reset the game each time the player clicks Play, we need to reset the 
game statistics, clear out the old aliens and bullets, build a new fleet, and 
center the ship, as shown here: 


# Reset the game statistics. 
self.stats.reset stats() 
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# Get rid of any remaining bullets and aliens. 
e self.bullets.empty() 
self.aliens.empty() 


# Create a new fleet and center the ship. 
e self. create fleet() 
self.ship.center ship() 


We reset the game statistics €, which gives the player three new ships. 
Then we set game active to True so the game will begin as soon as the code in 
this function finishes running. We empty the aliens and bullets groups 9, 
and then we create a new fleet and center the ship 9. 

Now the game will reset properly each time you click Play, allowing you 
to play it as many times as you want! 


Deactivating the Play Button 


One issue with our Play button is that the button region on the screen will 
continue to respond to clicks even when the Play button isn't visible. If you 
click the Play button area by accident after a game begins, the game will 
restart! 

To fix this, set the game to start only when game active is False: 


alien 
_invasion.py "Start a nen game whe cne player CiicK lc 
button clicked - self.play button.rect.collidepoint(mouse pos) 
e if button clicked and not self.game active: 


The flag button clicked stores a True or False value €, and the game will 
restart only if Play is clicked and the game is not currently active @. To test 
this behavior, start a new game and repeatedly click where the Play button 
should be. If everything works as expected, clicking the Play button area 
should have no effect on the gameplay. 


Hiding the Mouse Cursor 


We want the mouse cursor to be visible when the game is inactive, but 
once play begins, it just gets in the way. To fix this, we'll make it invisible 
when the game becomes active. We can do this at the end of the if block in 
check play button(): 


alien invasion.py 


# Hide the mouse cursor. 
pygame.mouse.set visible(False) 
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Passing False to set_visible() tells Pygame to hide the cursor when the 
mouse is over the game window. 

We’ll make the cursor reappear once the game ends so the player can 
click Play again to begin a new game. Here’s the code to do that: 


alien_invasion.py 


pygame -mouse.set_visible(True) 


We make the cursor visible again as soon as the game becomes inactive, 
which happens in _ship_hit(). Attention to details like this makes your game 
more professional looking and allows the player to focus on playing, rather 
than figuring out the user interface. 


TRY IT YOURSELF 


14-1. Press P to Play: Because Alien Invasion uses keyboard input to control the 
ship, it would be useful to start the game with a keypress. Add code that lets the 
player press P to start. It might help to move some code from | check play button() 
toa start game() method that can be called from check play button() and 

. check keydown events(). 


1422. Target Practice: Create a rectangle at the right edge of the screen that 
moves up and down at a steady rate. Then on the left side of the screen, cre- 
ate a ship that the player can move up and down while firing bullets at the 
rectangular target. Add a Play button that starts the game, and when the player 


misses the target three times, end the game and make the Play button reappear. 
Let the player restart the game with this Play button. 


Leveling Up 


In our current game, once a player shoots down the entire alien fleet, the 
player reaches a new level, but the game difficulty doesn't change. Let's 
liven things up a bit and make the game more challenging by increasing 
the game's speed each time a player clears the screen. 


Modifying the Speed Settings 


We'll first reorganize the Settings class to group the game settings into 
static and dynamic ones. We'll also make sure any settings that change 
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during the game reset when we start a new game. Here's the init () 
method for settings.py: 


Initialize the game's static settings. 


# How quickly the game speeds up 
self.speedup scale = 1.1 


self.initialize dynamic settings() 


We continue to initialize settings that stay constant in the — init () 
method. We add a speedup scale setting 6 to control how quickly the game 
speeds up: a value of 2 will double the game speed every time the player 
reaches a new level; a value of 1 will keep the speed constant. A value like 
1.1 should increase the speed enough to make the game challenging but 
not impossible. Finally, we call the initialize dynamic settings() method 
to initialize the values for attributes that need to change throughout the 
game 6. 

Here's the code for initialize dynamic settings(): 


def initialize dynamic settings(self): 
"""Initialize settings that change throughout the game. 
self.ship speed = 1.5 
self.bullet speed = 2.5 
self.alien speed - 1.0 


# fleet direction of 1 
self.fleet direction 


represents right; -1 represents left. 
1 


This method sets the initial values for the ship, bullet, and alien 
speeds. We'll increase these speeds as the player progresses in the game 
and reset them each time the player starts a new game. We include fleet 


. direction in this method so the aliens always move right at the beginning 


of a new game. We don't need to increase the value of fleet drop speed, 


because when the aliens move faster across the screen, they'll also come 
down the screen faster. 

To increase the speeds of the ship, bullets, and aliens each time the 
player reaches a new level, we'll write a new method called increase speed(): 


settings. py def increase_speed(self): 
"""Increase speed settings. 
self.ship speed *= self.speedup scale 
self.bullet speed *= self.speedup scale 
self.alien speed *= self.speedup scale 


To increase the speed of these game elements, we multiply each speed 
setting by the value of speedup scale. 

We increase the game's tempo by calling increase speed() in check 
bullet alien collisions() when the last alien in a fleet has been shot down: 


alien invasion.py 


self.settings. increase speed 0) 


Changing the values of the speed settings ship_speed, alien_speed, and 
bullet_speed is enough to speed up the entire game! 


Resetting the Speed 


Now we need to return any changed settings to their initial values each time 
the player starts a new game; otherwise, each new game would start with 
the increased speed settings of the previous game: 


alien_invasion.py 
# Reset the game settings. 
self.settings.initialize dynamic settings() 


Playing Alien Invasion should be more fun and challenging now. Each 
time you clear the screen, the game should speed up and become slightly 
more difficult. If the game becomes too difficult too quickly, decrease the 
value of settings.speedup scale. Or if the game isn't challenging enough, 
increase the value slightly. Find a sweet spot by ramping up the difficulty in 
a reasonable amount of time. The first couple of screens should be easy, the 
next few should be challenging but doable, and subsequent screens should 
be almost impossibly difficult. 
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TRY IT YOURSELF 


14-3. Challenging Target Practice: Start with your work from Exercise 14-2 
(page 283). Make the target move faster as the game progresses, and restart 
the target at the original speed when the player clicks Play. 


14-4. Difficulty Levels: Make a set of buttons for Alien Invasion that allows the 
player to select an appropriate starting difficulty level for the game. Each but- 


ton should assign the appropriate values for the attributes in Settings needed 
to create different difficulty levels. 


Scoring 


Let's implement a scoring system to track the game's score in real time and 
display the high score, level, and number of ships remaining. 
The score is a game statistic, so we'll add a score attribute to GameStats: 


game stats.py 


self.score = 0 


To reset the score each time a new game starts, we initialize score in 
reset stats() rather than init (). 


Displaying the Score 

To display the score on the screen, we first create a new class, Scoreboard. For 
now, this class will just display the current score. Eventually, we'll use it to 
report the high score, level, and number of ships remaining as well. Here's 
the first part of the class; save it as scoreboard.py: 


scoreboard.py ^ import pygame.font 


class Scoreboard: 
"""A class to report scoring information.""" 

e def init (self, ai game): 
"""Initialize scorekeeping attributes. 
self.screen - ai game.screen 
self.screen rect - self.screen.get rect() 
self.settings - ai game.settings 
self.stats - ai game.stats 


# Font settings for scoring information. 
self.text color - (30, 30, 30) 
self.font - pygame.font.SysFont(None, 48) 
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# Prepare the initial score image. 
self.prep score() 


Because Scoreboard writes text to the screen, we begin by importing the 
pygame.font module. Next, we give init ()the ai game parameter so it can 
access the settings, screen, and stats objects, which it will need to report the 
values we're tracking ®. Then we set a text color and instantiate a font 
object 6. 

To turn the text to be displayed into an image, we call prep score() O, 
which we define here: 


def prep score(self): 
"""Turn the score into a rendered image. 
score str - str(self.stats.score) 
self.score image - self.font.render(score str, True, 
self.text color, self.settings.bg color) 


# Display the score at the top right of the screen. 
self.score rect = self.score image.get rect() 
self.score rect.right - self.screen rect.right - 20 
self.score rect.top - 20 


In prep score(), we turn the numerical value stats.score into a string € 
and then pass this string to render(), which creates the image 6. To display 
the score clearly onscreen, we pass the screen's background color and the 
text color to render(). 

We'll position the score in the upper-right corner of the screen and 
have it expand to the left as the score increases and the width of the num- 
ber grows. To make sure the score always lines up with the right side of 
the screen, we create a rect called score rect 6 and set its right edge 20 pixels 
from the right edge of the screen O. We then place the top edge 20 pixels 
down from the top of the screen 8. 

Then we create a show score() method to display the rendered score 
image: 


def show score(self): 
"""Draw score to the screen. 
self.screen.blit(self.score image, self.score rect) 


This method draws the score image onscreen at the location score rect 
specifies. 


Making a Scoreboard 


To display the score, we'll create a Scoreboard instance in AlienInvasion. First, 
let's update the import statements: 


from scoreboard import Scoreboard 
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Next, we make an instance of Scoreboardin init (): 


alien_invasion.py def init (self): 
--snip-- 
pygame.display.set caption("Alien Invasion") 


# Create an instance to store game statistics, 
# — and create a scoreboard. 

self.stats - GameStats(self) 

self.sb - Scoreboard(self) 

--snip-- 


Then we draw the scoreboard onscreen in update screen(): 


alien invasion.py def update screen(self): 
--snip-- 
self.aliens.draw(self.screen) 


# Draw the score information. 
self.sb.show score() 


# Draw the play button if the game is inactive. 
--snip-- 


We call show score() just before we draw the Play button. 

When you run Alien Invasion now, a 0 should appear at the top right of 
the screen. (At this point, we just want to make sure the score appears in 
the right place before developing the scoring system further.) Figure 14-2 
shows the score as it appears before the game starts. 

Next, we'll assign point values to each alien! 


2 


Figure 14-2: The score appears at the top-right corner of the screen. 
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Updating the Score as Aliens Are Shot Down 


To write a live score onscreen, we update the value of stats.score whenever 
an alien is hit, and then call prep_score() to update the score image. But 
first, let's determine how many points a player gets each time they shoot 
down an alien: 


settings.py def initialize dynamic settings( 


# Scoring settings 
self.alien points - 50 


We'll increase each alien's point value as the game progresses. To make 
sure this point value is reset each time a new game starts, we set the value in 
initialize dynamic settings(). 

Let's update the score in. check bullet alien collisions() each time an 
alien is shot down: 


alien invasion.py 


if collisions: 
self.stats.score += self.settings.alien points 
self.sb.prep score() 


When a bullet hits an alien, Pygame returns a collisions dictionary. 
We check whether the dictionary exists, and if it does, the alien's value is 
added to the score. We then call prep score() to create a new image for the 
updated score. 

Now when you play Alien Invasion, you should be able to rack up points! 


Resetting the Score 


Right now, we're only prepping a new score after an alien has been hit, 
which works for most of the game. But when we start a new game, we'll still 
see our score from the old game until the first alien is hit. 

We can fix this by prepping the score when starting a new game: 


alien invasion.py 


self.sb.prep score() 
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We call prep score() after resetting the game stats when starting a new 
game. This preps the scoreboard with a score of 0. 


Making Sure to Score All Hits 


As currently written, our code could miss scoring for some aliens. For example, 
if two bullets collide with aliens during the same pass through the loop or 
if we make an extra-wide bullet to hit multiple aliens, the player will only 
receive points for hitting one of the aliens. To fix this, let's refine the way 
that bulletalien collisions are detected. 

In check bullet alien collisions(), any bullet that collides with an alien 
becomes a key in the collisions dictionary. The value associated with each 
bullet is a list of aliens it has collided with. We loop through the values in 
the collisions dictionary to make sure we award points for each alien hit: 


f 


for aliens in collisions.values(): 
self.stats.score += self.settings.alien_points * len(aliens) 


If the collisions dictionary has been defined, we loop through all values 
in the dictionary. Remember that each value is a list of aliens hit by a single 
bullet. We multiply the value of each alien by the number of aliens in each list 
and add this amount to the current score. To test this, change the width of 
a bullet to 300 pixels and verify that you receive points for each alien you hit 
with your extra-wide bullets; then return the bullet width to its normal value. 


Increasing Point Values 


Because the game gets more difficult each time a player reaches a new level, 
aliens in later levels should be worth more points. To implement this func- 
tionality, we'll add code to increase the point value when the game's speed 
increases: 


# How quickly the alien point values increase 
self.score scale = 1.5 


settings.py 


scoreboard.py 


Increase speed settings and alien point values. 


self.alien points - int(self.alien points * self.score scale) 


We define a rate at which points increase, which we call score scale 6». 
A small increase in speed (1.1) makes the game more challenging quickly. 
But to see a more notable difference in scoring, we need to change the alien 
point value by a larger amount (1.5). Now when we increase the game's 
speed, we also increase the point value of each hit 6. We use the int() func- 
tion to increase the point value by whole integers. 

To see the value of each alien, add a print() call to the increase speed() 
method in Settings: 


print(self. alien points) 


The new point value should appear in the terminal every time you 
reach a new level. 


Be sure to remove the print() call after verifying that the point value is increasing, or 
it might affect your game's performance and distract the player. 


Rounding the Score 


Most arcade-style shooting games report scores as multiples of 10, so let’s fol- 
low that lead with our scores. Also, let’s format the score to include comma 
separators in large numbers. We'll make this change in Scoreboard: 


rounded score = round(self.stats.score, -1) 
score str = f'(rounded score:,]" 


The round() function normally rounds a float to a set number of deci- 
mal places given as the second argument. However, when you pass a nega- 
tive number as the second argument, round() will round the value to the 
nearest 10, 100, 1,000, and so on. This code tells Python to round the value 
of stats.score to the nearest 10 and assign it to rounded score. 

We then use a format specifier in the f-string for the score. A format 
specifier is a special sequence of characters that modifies the way a vari- 
able's value is presented. In this case the sequence :, tells Python to insert 
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commas at appropriate places in the numerical value that’s provided. This 
results in strings like 1,000,000 instead of 1000000. 

Now when you run the game, you should see a neatly formatted, rounded 
score even when you rack up lots of points, as shown in Figure 14-3. 
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Figure 14-3: A rounded score with comma separators 


High Scores 


Every player wants to beat a game’s high score, so let’s track and report high 
scores to give players something to work toward. We'll store high scores in 
GameStats: 


game_stats.py def init (self, ai; 
^ snip- E: 
# High score should never be reset. 
self.high score = 0 


Because the high score should never be reset, we initialize high_score in 
. init ()rather than in reset_stats(). 

Next, we'll modify Scoreboard to display the high score. Let's start with 
the init () method: 


scoreboard.py def init (self, ai game): 
--snip-- 
# Prepare the initial score images. 
self.prep score() 


e self.prep high score() 


The high score will be displayed separately from the score, so we need a 
new method, prep high score(), to prepare the high-score image 6. 
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Here's the prep high score() method: 


scoreboard.py def prep high score(self): 
"""Turn the high score into a rendered image.""" 
e high score - round(self.stats.high score, -1) 
high score str = f"{high score:,]" 
e self.high score image - self.font.render(high score str, True, 


self.text color, self.settings.bg color) 


# Center the high score at the top of the screen. 
self.high score rect - self.high score image.get rect() 


e self.high score rect.centerx = self.screen rect.centerx 
o self.high score rect.top - self.score rect.top 
We round the high score to the nearest 10 and format it with com- 
mas 6.. We then generate an image from the high score @, center the 
high score rect horizontally ©, and set its top attribute to match the top 
of the score image O. 
The show score() method now draws the current score at the top right 
and the high score at the top center of the screen: 
scoreboard.py 
self.screen.blit(self.high score image, self.high score rect) 
To check for high scores, we'll write a new method, check high score(), 
in Scoreboard: 
scoreboard.py def check high score(self): 


Check to see if there's a new high score. 
if self.stats.score » self.stats.high score: 
self.stats.high score - self.stats.score 
self.prep high score() 


The method check high score() checks the current score against the 
high score. If the current score is greater, we update the value of high score 
and call prep high score() to update the high score's image. 

We need to call check high score() each time an alien is hit after updat- 
ing the score in. check bullet alien collisions(): 


alien invasion.py 


self.sb. check high score 0 


We call check high score() when the collisions dictionary is present, and 
we do so after updating the score for all the aliens that have been hit. 
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The first time you play Alien Invasion, your score will be the high score, 
so it will be displayed as the current score and the high score. But when you 
start a second game, your high score should appear in the middle and your 
current score should appear at the right, as shown in Figure 14-4. 
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Figure 14-4: The high score is shown at the top center of the screen. 


Displaying the Level 
To display the player's level in the game, we first need an attribute in 


GameStats representing the current level. To reset the level at the start of 
each new game, initialize itin reset stats(): 


def reset stats(self): 
"""Initialize statistics that can change during the game. 
self.ships left = self.settings.ship limit 
self.score = 0 
self.level = 1 


nun 


To have Scoreboard display the current level, we call a new method, 
prep level() from init (): 


def init (self, ai game): 
--snip-- 
self.prep high score() 
self.prep level() 


Here's prep level(): 


def prep level(self): 
"""Turn the level into a rendered image. 
level str - str(self.stats.level) 


e self.level image - self.font.render(level str, True, 
self.text color, self.settings.bg color) 


# Position the level below the score. 

self.level rect - self.level image.get rect() 
self.level rect.right - self.score rect.right 
self.level rect.top = self.score rect.bottom + 10 


o0 


The prep level() method creates an image from the value stored in 
stats.level 60 and sets the image’s right attribute to match the score's right 
attribute @. It then sets the top attribute 10 pixels beneath the bottom of 
the score image to leave space between the score and the level 8. 

We also need to update show score(): 


scoreboard.py c 0 


Draw scores and level to the screen. 


self.screen.blit(self.level image, self.level rect) 


This new line draws the level image to the screen. 
We'll increment stats.level and update the level image in. check bullet 
_alien_collisions(): 


alien_invasion.py 


# Increase level. 
self.stats.level += 1 
self.sb.prep level() 


If a fleet is destroyed, we increment the value of stats.level and call 
prep level() to make sure the new level displays correctly. 

To ensure the level image updates properly at the start of a new game, 
we also call prep level() when the player clicks the Play button: 


alien invasion.py 


self.sb. prep level() 


We call prep level() right after calling prep score(). 
Now you'll see how many levels you've completed, as shown in 
Figure 14-5. 


Scoring 295 


eo Alien invasion 


3,589,080 51,650 


Bo A iaa 

e B A 

5 a 
as 


» »»»» 
» »»»» 
» »»»» 
» P PPP. 


a 


a 


Figure 14-5: The current level appears just below the current score. 


In some classic games, the scores have labels, such as Score, High Score, and Level. 
We've omitted these labels because the meaning of each number becomes clear once 
you've played the game. To include these labels, add them to the score strings just 
before the calls to font.render() in Scoreboard. 


Displaying the Number of Ships 


Finally, let’s display the number of ships the player has left, but this time, 
let’s use a graphic. To do so, we’ll draw ships in the upper-left corner of 
the screen to represent how many ships are left, just as many classic arcade 
games do. 

First, we need to make Ship inherit from Sprite so we can create a group 
of ships: 


ship.py ^ import pygame 
from pygame.sprite import Sprite 


@ class Ship(Sprite): 
"""A class to manage the ship.""" 


def init (self, ai game): 
"""Initialize the ship and set its starting position.""" 
e super(). init () 
--snip-- 


Here we import Sprite, make sure Ship inherits from Sprite 0, and call 
super()atthe beginning of init () @. 
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Next, we need to modify Scoreboard to create a group of ships we can 
display. Here are the import statements for Scoreboard: 


scoreboard. py ort pygame.fo 
from pygame.sprite import Group 


from ship import Ship 


Because we’re making a group of ships, we import the Group and Ship 
classes. 
Here's init (): 


scoreboard.py 


self.ai game - ai game 


self.prep ships() 


We assign the game instance to an attribute, because we'll need it to 
create some ships. We call prep ships() after the call to prep level(). 
Here's prep ships(): 


scoreboard.py def prep ships(self): 
"""Show how many ships are left. 
self.ships - Group() 
for ship number in range(self.stats.ships left): 
ship - Ship(self.ai game) 
ship.rect.x = 10 + ship number * ship.rect.width 
ship.rect.y - 10 
self.ships.add(ship) 


0o00 00 


The prep_ships() method creates an empty group, self.ships, to hold 
the ship instances ®. To fill this group, a loop runs once for every ship the 
player has left 6. Inside the loop, we create a new ship and set each ship's 
x-coordinate value so the ships appear next to each other with a 10-pixel 
margin on the left side of the group of ships ®. We set the y-coordinate 
value 10 pixels down from the top of the screen so the ships appear in the 
upper-left corner of the screen O. Then we add each new ship to the group 
ships 9. 

Now we need to draw the ships to the screen: 


scoreboard.py ef sho 


Draw scores, level, and ships to the screen. 


self.ships.draw(self.screen) 
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To display the ships on the screen, we call draw() on the group, and 
Pygame draws each ship. 

To show the player how many ships they have to start with, we call 
prep ships() when a new game starts. We do this in check play button() in 


AlienInvasion: 
alien_invasion.py def check play button(self, mouse pos): 
--snip-- 
if button clicked and not self.game active: 
--snip-- 


self.sb.prep level() 
self.sb.prep ships() 
--snip-- 


We also call prep ships() when a ship is hit, to update the display of ship 
images when the player loses a ship: 


alien invasion.py def ship hit(self): 
"""Respond to ship being hit by alien. 
if self.stats.ships left » 0: 
# Decrement ships left, and update scoreboard. 
self.stats.ships left -= 1 
self.sb.prep ships() 
--snip-- 


We call prep ships() after decreasing the value of ships left, so the cor- 
rect number of remaining ships displays each time a ship is destroyed. 

Figure 14-6 shows the complete scoring system, with the remaining 
ships displayed at the top left of the screen. 
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Figure 14-6: The complete scoring system for Alien Invasion 
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Summary 


TRY IT YOURSELF 


14-5. All-Time High Score: The high score is reset every time a player closes 
and restarts Alien Invasion. Fix this by writing the high score to a file before 
calling sys.exit() and reading in the high score when initializing its value in 
GameStats. 


14-6. Refactoring: Look for methods that are doing more than one task, and 
refactor them to organize your code and make it efficient. For example, move 
some of the code in check bullet alien collisions(), which starts a new 
level when the fleet of aliens has been destroyed, to a function called start 
. new level(). Also, move the four separate method calls in the — init () method 
in Scoreboard to a method called prep images() to shorten — init (). The prep 
_images() method could also help simplify check play button() or start game() 
if you've already refactored check play button(). 


Before attempting to refactor the project, see Appendix D to learn 
how to restore the project to a working state if you introduce bugs 
while refactoring. 


14-7. Expanding the Game: Think of a way to expand Alien Invasion. For 
example, you could program the aliens to shoot bullets down at your ship. You 
can also add shields for your ship to hide behind, which can be destroyed by 
bullets from either side. Or you can use something like the pygame.mixer module 
to add sound effects, such as explosions and shooting sounds. 


14-8. Sideways Shooter, Final Version: Continue developing Sideways Shooter, 
using everything we've done in this project. Add a Play button, make the game 
speed up at appropriate points, and develop a scoring system. Be sure to refac- 
tor your code as you work, and look for opportunities to customize the game 
beyond what has been shown in this chapter. 


In this chapter, you learned how to implement a Play button to start a new 
game. You also learned how to detect mouse events and hide the cursor in 
active games. You can use what you've learned to create other buttons, like a 
Help button to display instructions on how to play your games. You also learned 
how to modify the speed of a game as it progresses, implement a progressive 
scoring system, and display information in textual and nontextual ways. 
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GENERATING DATA 


Data visualization is the use of visual repre- 

sentations to explore and present patterns 
in datasets. It’s closely associated with data 

analysis, which uses code to explore the pat- 

terns and connections in a dataset. A dataset can be a 
small list of numbers that fits in a single line of code, 
or it can be terabytes of data that include many differ- 
ent kinds of information. 


Creating effective data visualizations is about more than just making 
information look nice. When a representation of a dataset is simple and 
visually appealing, its meaning becomes clear to viewers. People will see 
patterns and significance in your datasets that they never knew existed. 

Fortunately, you don’t need a supercomputer to visualize complex data. 
Python is so efficient that with just a laptop, you can quickly explore data- 
sets containing millions of individual data points. These data points don’t 
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have to be numbers; with the basics you learned in the first part of this 
book, you can analyze non-numerical data as well. 

People use Python for data-intensive work in genetics, climate research, 
political and economic analysis, and much more. Data scientists have writ- 
ten an impressive array of visualization and analysis tools in Python, many of 
which are available to you as well. One of the most popular tools is Matplotlib, 
a mathematical plotting library. In this chapter, we'll use Matplotlib to make 
simple plots, such as line graphs and scatter plots. Then we'll create a more 
interesting dataset based on the concept of a random walk—a visualization 
generated from a series of random decisions. 

We'll also use a package called Plotly, which creates visualizations that 
work well on digital devices, to analyze the results of rolling dice. Plotly gener- 
ates visualizations that automatically resize to fit a variety of display devices. 
These visualizations can also include a number of interactive features, such as 
emphasizing particular aspects of the dataset when users hover over different 
parts of the visualization. Learning to use Matplotlib and Plotly will help you 
get started visualizing the kinds of data you’re most interested in. 


Installing Matplotlib 


To use Matplotlib for your initial set of visualizations, you'll need to install 
it using pip, just like we did with pytest in Chapter 11 (see "Installing pytest 
with pip" on page 210). 

To install Matplotlib, enter the following command at a terminal prompt: 


$ python -m pip install --user matplotlib 


If you use a command other than python to run programs or start a ter- 
minal session, such as python3, your command will look like this: 


$ python3 -m pip install --user matplotlib 


To see the kinds of visualizations you can make with Matplotlib, visit the 
Matplotlib home page at https://matplotlib.org and click Plot types. When you 
click a visualization in the gallery, you'll see the code used to generate the plot. 


Plotting a Simple Line Graph 
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Let's plot a simple line graph using Matplotlib and then customize it to 
create a more informative data visualization. We'll use the square number 
sequence 1, 4, 9, 16, and 25 as the data for the graph. 

To make a simple line graph, specify the numbers you want to work 
with and let Matplotlib do the rest: 


import matplotlib.pyplot as plt 


squares - [1, 4, 9, 16, 25] 


@ fig, ax = plt.subplots() 
ax.plot (squares) 


plt.show() 


We first import the pyplot module using the alias plt so we don’t have to 
type pyplot repeatedly. (You'll see this convention often in online examples, 
so we'll use it here.) The pyplot module contains a number of functions that 
help generate charts and plots. 

We create a list called squares to hold the data that we'll plot. Then we 
follow another common Matplotlib convention by calling the subplots() 
function 6. This function can generate one or more plots in the same 
figure. The variable fig represents the entire figure, which is the collection 
of plots that are generated. The variable ax represents a single plot in the 
figure; this is the variable we'll use most of the time when defining and cus- 
tomizing a single plot. 

We then use the plot() method, which tries to plot the data it's given in 
a meaningful way. The function plt.show() opens Matplotlib’s viewer and 
displays the plot, as shown in Figure 15-1. The viewer allows you to zoom 
and navigate the plot, and you can save any plot images you like by clicking 
the disk icon. 
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Figure 15-1: One of the simplest plots you can make in Matplotlib 


Changing the Label Type and Line Thickness 


Although the plot in Figure 15-1 shows that the numbers are increasing, the 
label type is too small and the line is a little thin to read easily. Fortunately, 
Matplotlib allows you to adjust every feature of a visualization. 
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We'll use a few of the available customizations to improve this plot's 
readability. Let’s start by adding a title and labeling the axes: 


squares = [41 1 QO 146 Ot 
Squares 1, 4, 9 16, 25] 


fig, ax = plt . subplots ( ) 


ales CN 


@ ax. plot(squares, linewidth=3) 


# Set chart title and label axes. 
@ ax.set title("Square Numbers", fontsize=24) 
© ax.set_xlabel("Value", fontsize=14) 
ax.set_ylabel("Square of Value", fontsize=14) 


# Set size of tick labels. 
O ax.tick params(labelsize-14) 


The linewidth parameter controls the thickness of the line that plot() 
generates 0. Once a plot has been generated, there are many methods 
available to modify the plot before it's presented. The set title() method 
sets an overall title for the chart 6. The fontsize parameters, which appear 
repeatedly throughout the code, control the size of the text in various ele- 
ments on the chart. 

The set xlabel() and set ylabel() methods allow you to set a title for each 
of the axes 6, and the method tick params() styles the tick marks O. Here 
tick params() sets the font size of the tick mark labels to 14 on both axes. 

As you can see in Figure 15-2, the resulting chart is much easier to read. 
The label type is bigger, and the line graph is thicker. It's often worth experi- 
menting with these values to see what works best in the resulting graph. 
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Figure 15-2: The chart is much easier to read now. 
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Correcting the Plot 


Now that we can read the chart better, we can see that the data is not plot- 
ted correctly. Notice at the end of the graph that the square of 4.0 is shown 
as 25! Let's fix that. 

When you give plot() a single sequence of numbers, it assumes the first 
data point corresponds to an x-value of 0, but our first point corresponds to 
an x-value of 1. We can override the default behavior by giving plot() both 
the input and output values used to calculate the squares: 


import matplotlib.pyplot as plt 


input_values = [1, 2, 3, 4, 5] 


squares (1, 4, 9, 16, 25] 


fig, ax = plt.subplots() 


ax.plot(input_values, squares, linewidth=3) 


hart title and label axes. 


Now plot() doesn’t have to make any assumptions about how the out- 
put numbers were generated. The resulting plot, shown in Figure 15-3, is 
correct. 
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Figure 15-3: The data is now plotted correctly. 


You can specify a number of arguments when calling plot() and use a 
number of methods to customize your plots after generating them. We’ll 
continue to explore these approaches to customization as we work with 
more interesting datasets throughout this chapter. 


Generating Data 305 


mpl_squares.py 


306 


Chapter 15 


Using Built-in Styles 

Matplotlib has a number of predefined styles available. These styles contain 
a variety of default settings for background colors, gridlines, line widths, 
fonts, font sizes, and more. They can make your visualizations appealing 
without requiring much customization. To see the full list of available styles, 
run the following lines in a terminal session: 


>>> import matplotlib.pyplot as plt 

>>> plt.style.available 

['Solarize Light2', ' classic test patch', 
--snip-- 


 mpl-gallery', 


To use any of these styles, add one line of code before calling subplots(): 


plt.style.use('seaborn') 


This code generates the plot shown in Figure 15-4. A wide variety of 
styles is available; play around with these styles to find some that you like. 
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Figure 15-4: The built-in seaborn style 


Plotting and Styling Individual Points with scatter() 


Sometimes, it’s useful to plot and style individual points based on certain 
characteristics. For example, you might plot small values in one color and 
larger values in a different color. You could also plot a large dataset with 
one set of styling options and then emphasize individual points by replot- 
ting them with different options. 


scatter 
_squares.py 


To plot a single point, pass the single x- and y-values of the point to 
scatter(): 


import matplotlib.pyplot as plt 
plt.style.use('seaborn') 

fig, ax = plt.subplots() 
ax.scatter(2, 4) 


plt.show() 


Let's style the output to make it more interesting. We'll add a title, label 
the axes, and make sure all the text is large enough to read: 


t 


(1) ax.scatter(2, 4, s=200) 


# Set chart title and label axes. 
ax.set_title("Square Numbers", fontsize=24) 
ax.set_xlabel("Value", fontsize=14) 
ax.set_ylabel("Square of Value", fontsize=14) 


# Set size of tick labels. 
ax.tick params(labelsize-14) 


We call scatter() and use the s argument to set the size of the dots used 
to draw the graph 6. When you run scatter. squares.py now, you should see a 
single point in the middle of the chart, as shown in Figure 15-5. 
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Figure 15-5: Plotting a single point 
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Plotting a Series of Points with scatter() 


To plot a series of points, we can pass scatter() separate lists of x- and 
y-values, like this: 


scatter 
_squares.py 
x values = [1, 2, 3, 4, 5] 
y values = [1, 4, 9, 16, 25] 
ax. scatter(x values, y values, $=100) 

The x_values list contains the numbers to be squared, and y_values contains 
the square of each number. When these lists are passed to scatter(), Matplotlib 
reads one value from each list as it plots each point. The points to be plotted 
are (1, 1), (2, 4), (3, 9), (4, 16), and (5, 25); Figure 15-6 shows the result. 
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Figure 15-6: A scatter plot with multiple points 
Calculating Data Automatically 
Writing lists by hand can be inefficient, especially when we have many 
points. Rather than writing out each value, let's use a loop to do the cal- 
culations for us. 
Here's how this would look with 1,000 points: 
scatter 
_squares.py 
@ x values = range(1, 1001) 


y values = [x**2 for x in x values] 
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@ ax. scatter(x 


t chart title and label axes. 


# Set the range for each axis. 
© ax.axis([0, 1100, 0, 1 100 000]) 


plt.show() 


We start with a range of x-values containing the numbers 1 through 
1,000 €. Next, a list comprehension generates the y-values by looping 
through the x-values (for x in x values), squaring each number (x**2), and 
assigning the results to y values. We then pass the input and output lists to 
scatter() @. Because this is a large dataset, we use a smaller point size. 

Before showing the plot, we use the axis() method to specify the 
range of each axis ©. The axis() method requires four values: the mini- 
mum and maximum values for the x-axis and the y-axis. Here, we run 
the x-axis from 0 to 1,100 and the y-axis from 0 to 1,100,000. Figure 15-7 
shows the result. 
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Figure 15-7: Python can plot 1,000 points as easily as it plots 5 points. 


Customizing Tick Labels 


When the numbers on an axis get large enough, Matplotlib defaults to 
scientific notation for tick labels. This is usually a good thing, because 
larger numbers in plain notation take up a lot of unnecessary space on a 
visualization. 
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scatter 
_Squares.py 


Chapter 15 


Almost every element of a chart is customizable, so you can tell Matplotlib 
to keep using plain notation if you prefer: 


ax.ticklabel format(style- 'plain') 


The ticklabel format() method allows you to override the default tick 
label style for any plot. 


Defining Custom Colors 


To change the color of the points, pass the argument color to scatter() with 
the name of a color to use in quotation marks, as shown here: 


ax.scatter(x values, y values, color-'red', s=10) 


You can also define custom colors using the RGB color model. To 
define a color, pass the color argument a tuple with three float values (one 
each for red, green, and blue, in that order), using values between 0 and 1. 
For example, the following line creates a plot with light-green dots: 


ax.scatter(x values, y values, color-(0, 0.8, 0), s-10) 


Values closer to 0 produce darker colors, and values closer to 1 produce 
lighter colors. 


Using a Colormap 


A colormap is a sequence of colors in a gradient that moves from a starting 
to an ending color. In visualizations, colormaps are used to emphasize pat- 
terns in data. For example, you might make low values a light color and 
high values a darker color. Using a colormap ensures that all points in the 
visualization vary smoothly and accurately along a well-designed color scale. 

The pyplot module includes a set of built-in colormaps. To use one of 
these colormaps, you need to specify how pyplot should assign a color to 
each point in the dataset. Here's how to assign a color to each point, based 
on its y-value: 


ax.scatter(x values, y values, c-y values, cmap=plt.cm.Blues, s-10) 


The c argument is similar to color, but it's used to associate a sequence 
of values with a color mapping. We pass the list of y-values to c, and then 


tell pyplot which colormap to use with the cmap argument. This code colors 
the points with lower y-values light blue and the points with higher y-values 
dark blue. Figure 15-8 shows the resulting plot. 


You can see all the colormaps available in pyplot at https://matplotlib.org. Go to 
Tutorials, scroll down to Colors, and click Choosing Colormaps in Matplotlib. 
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Figure 15-8: A plot using the Blues colormap 


Saving Your Plots Automatically 


If you want to save the plot to a file instead of showing it in the Matplotlib 
viewer, you can use plt.savefig() instead of plt.show(): 


plt.savefig('squares plot.png', bbox inches-'tight') 


The first argument is a filename for the plot image, which will be saved 
in the same directory as scatter squares.py. The second argument trims extra 
whitespace from the plot. If you want the extra whitespace around the plot, 
you can omit this argument. You can also call savefig() with a Path object, 
and write the output file anywhere you want on your system. 


TRY IT YOURSELF 


15-1. Cubes: A number raised to the third power is a cube. Plot the first five 


cubic numbers, and then plot the first 5,000 cubic numbers. 


15-2. Colored Cubes: Apply a colormap to your cubes plot. 
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Random Walks 


In this section, we'll use Python to generate data for a random walk and 
then use Matplotlib to create a visually appealing representation of that 
data. A random walk is a path that’s determined by a series of simple deci- 
sions, each of which is left entirely to chance. You might imagine a random 
walk as the path a confused ant would take if it took every step in a ran- 
dom direction. 

Random walks have practical applications in nature, physics, biology, 
chemistry, and economics. For example, a pollen grain floating on a drop 
of water moves across the surface of the water because it’s constantly pushed 
around by water molecules. Molecular motion in a water drop is random, 
so the path a pollen grain traces on the surface is a random walk. The code 
we'll write next models many real-world situations. 


Creating the RandomWalk Class 


To create a random walk, we'll create a RandomWalk class, which will make ran- 
dom decisions about which direction the walk should take. The class needs 
three attributes: one variable to track the number of points in the walk, and 
two lists to store the x- and y-coordinates of each point in the walk. 

We'll only need two methods for the RandomWalk class: the — init () 
method and fill walk(), which will calculate the points in the walk. Let's 
start with the init () method: 


random ® from random import choice 


_walk.py 


random 
_walk.py 
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class RandomWalk: 
"""A class to generate random walks.""" 
def init (self, num points-5000): 
"""Initialize attributes of a walk.""" 
self.num points - num points 


# All walks start at (0, 0). 
self.x values - [0] 
self.y values - [0] 


To make random decisions, we'll store possible moves in a list and use 
the choice() function (from the random module) to decide which move to 
make each time a step is taken 6. We set the default number of points in 
a walk to 5000, which is large enough to generate some interesting patterns 
but small enough to generate walks quickly 0. Then we make two lists to 
hold the x- and y-values, and we start each walk at the point (0, 0) 8. 


Choosing Directions 


We'll use the fill walk() method to determine the full sequence of points 
in the walk. Add this method to random, walk.py: 


def fill walk(self): 
"""Calculate all the points in the walk.""" 


rw_visual.py 


# Keep taking steps until the walk reaches the desired length. 
while len(self.x_values) < self.num_points: 


# Decide which direction to go, and how far to go. 
x direction = choice([1, -1]) 

x distance = choice([O, 1, 2, 3, 4]) 

X step = x direction * x distance 


y direction = choice([1, -1]) 
y distance - choice([O, 1, 2, 3, 4]) 
y step = y direction * y distance 


# Reject moves that go nowhere. 
if x step -- O and y step -- 
continue 


# Calculate the new position. 
x = self.x_values[-1] + x_step 
y = self.y values[-1] + y step 


self.x values.append(x) 
self.y values.append(y) 


We first set up a loop that runs until the walk is filled with the correct 
number of points 60. The main part of fill walk() tells Python how to simu- 
late four random decisions: Will the walk go right or left? How far will it go 
in that direction? Will it go up or down? How far will it go in that direction? 

We use choice([1, -1]) to choose a value for x direction, which returns 
either 1 for movement to the right or -1 for movement to the left 6. Next, 
choice([0, 1, 2, 3, 4]) randomly selects a distance to move in that direc- 
tion. We assign this value to x distance. The inclusion of a 0 allows for the 
possibility of steps that have movement along only one axis. 

We determine the length of each step in the x- and y-directions by mul- 
tiplying the direction of movement by the distance chosen 6 @. A positive 
result for x step means move to the right, a negative result means move 
to the left, and 0 means move vertically. A positive result for y step means 
move up, negative means move down, and 0 means move horizontally. If the 
values of both x step and y step are 0, the walk doesn't go anywhere; when 
this happens, we continue the loop 9. 

To get the next x-value for the walk, we add the value in x step to the 
last value stored in x values O and do the same for the y-values. When we 
have the new point's coordinates, we append them to x values and y values. 


Plotting the Random Walk 


Here's the code to plot all the points in the walk: 


import matplotlib.pyplot as plt 
from random walk import RandomWalk 


# Make a random walk. 
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@ rw = RandomWalk() 


rw. fill_walk() 


# Plot the points in the walk. 
plt.style.use('classic') 
fig, ax = plt.subplots() 


@ ax.scatter(rw.x values, rw.y values, s=15) 
© ax.set aspect('equal') 


rw. visual.py 
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plt.show() 


We begin by importing pyplot and RandomWalk. We then create a random 
walk and assign it to rw €, making sure to call fill walk(). To visualize the walk, 
we feed the walk's x- and y-values to scatter() and choose an appropriate 
dot size 0. By default, Matplotlib scales each axis independently. But that 
approach would stretch most walks out horizontally or vertically. Here we 
use the set aspect() method to specify that both axes should have equal 
spacing between tick marks 8. 

Figure 15-9 shows the resulting plot with 5,000 points. The images in 
this section omit Matplotlib's viewer, but you'll continue to see it when you 
run ru visual.py. 
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Figure 15-9: A random walk with 5,000 points 


Generating Multiple Random Walks 


Every random walk is different, and it's fun to explore the various patterns 
that can be generated. One way to use the preceding code to make multiple 
walks without having to run the program several times is to wrap itin a 
while loop, like this: 


import matplotlib.pyplot as plt 
from random walk import RandomWalk 


# Keep making new walks, as long as the program is active. 


rw_visual.py 


LU 


while True: 


keep running - input("Make another walk? (y/n): ") 


if keep running == 'n': 
break 


This code generates a random walk, displays it in Matplotlib's viewer, 
and pauses with the viewer open. When you close the viewer, you'll be asked 
whether you want to generate another walk. If you generate a few walks, 
you should see some that stay near the starting point, some that wander 
off mostly in one direction, some that have thin sections connecting larger 
groups of points, and many other kinds of walks. When you want to end the 
program, press N. 


Styling the Walk 


In this section, we'll customize our plots to emphasize the important char- 
acteristics of each walk and deemphasize distracting elements. To do so, we 
identify the characteristics we want to emphasize, such as where the walk 
began, where it ended, and the path taken. Next, we identify the character- 
istics to deemphasize, such as tick marks and labels. The result should be 
a simple visual representation that clearly communicates the path taken in 
each random walk. 


Coloring the Points 


We'll use a colormap to show the order of the points in the walk, and 
remove the black outline from each dot so the color of the dots will be 
clearer. To color the points according to their position in the walk, we 
pass the c argument a list containing the position of each point. Because 
the points are plotted in order, this list just contains the numbers from 0 
to 4,999: 


point numbers - range(rw.num points) 
ax.scatter(rw.x values, rw.y values, c-point numbers, cmap-plt.cm.Blues, 
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We use range() to generate a list of numbers equal to the number of 
points in the walk 0. We assign this list to point numbers, which we'll use to 
set the color of each point in the walk. We pass point numbers to the c argu- 
ment, use the Blues colormap, and then pass edgecolors-'none' to get rid of 
the black outline around each point. The result is a plot that varies from 
light to dark blue, showing exactly how the walk moves from its starting 
point to its ending point. This is shown in Figure 15-10. 
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Figure 15-10: A random walk colored with the Blues colormap 


Plotting the Starting and Ending Points 


In addition to coloring points to show their position along the walk, it 
would be useful to see exactly where each walk begins and ends. To do so, 
we can plot the first and last points individually after the main series has 
been plotted. We'll make the end points larger and color them differently 
to make them stand out: 


rw. visud.py | --sni 
while True: 
--snip-- 


ax.scatter(rw.x values, rw.y values, c-point numbers, cmap-plt.cm.Blues, 


edgecolors-'none', s-15) 
ax.set aspect('equal') 


# Emphasize the first and last points. 

ax.scatter(0, 0, c-'green', edgecolors-'none', s-100) 

ax.scatter(rw.x values[-1], rw.y values[-1], c='red', edgecolors-'none', 
s=100) 


To show the starting point, we plot the point (0, 0) in green and in a 
larger size (s=100) than the rest of the points. To mark the end point, we 


316 Chapter 15 


rw_visual.py 


rw_visual.py 


plot the last x- and y-values in red with a size of 100 as well. Make sure you 
insert this code just before the call to plt.show() so the starting and ending 
points are drawn on top of all the other points. 

When you run this code, you should be able to spot exactly where each 
walk begins and ends. If these end points don’t stand out clearly enough, 
adjust their color and size until they do. 


Cleaning Up the Axes 


Let’s remove the axes in this plot so they don’t distract from the path of 
each walk. Here’s how to hide the axes: 


# Remove the axes. 
ax.get_xaxis().set_visible(False) 
ax.get_yaxis().set_visible(False) 


To modify the axes, we use the ax.get_xaxis() and ax.get_yaxis() meth- 
ods to get each axis, and then chain the set_visible() method to make 
each axis invisible. As you continue to work with visualizations, you'll fre- 
quently see this chaining of methods to customize different aspects of a 
visualization. 

Run rw visual.py now; you should see a series of plots with no axes. 


Adding Plot Points 


Let's increase the number of points, to give us more data to work with. 
To do so, we increase the value of num points when we make a RandomWalk 
instance and adjust the size of each dot when drawing the plot: 


rw - RandomWalk(50 000) 


ax.scatter(rw.x values, rw.y values, c-point numbers, cmap=plt.cm.Blues, 
edgecolors-'none', s-1) 
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This example creates a random walk with 50,000 points and plots each 
point at size s=1. The resulting walk is wispy and cloudlike, as shown in 
Figure 15-11. We’ve created a piece of art from a simple scatter plot! 

Experiment with this code to see how much you can increase the num- 
ber of points in a walk before your system starts to slow down significantly 
or the plot loses its visual appeal. 


Figure 15-11: A walk with 50,000 points 


Altering the Size to Fill the Screen 


A visualization is much more effective at communicating patterns in data 
if it fits nicely on the screen. To make the plotting window better fit your 
screen, you can adjust the size of Matplotlib’s output. This is done in the 
subplots() call: 


fig, ax = plt.subplots(figsize=(15, 9)) 


When creating a plot, you can pass subplots() a figsize argument, which 
sets the size of the figure. The figsize parameter takes a tuple that tells 
Matplotlib the dimensions of the plotting window in inches. 

Matplotlib assumes your screen resolution is 100 pixels per inch; if this 
code doesn’t give you an accurate plot size, adjust the numbers as necessary. 
Or, if you know your system’s resolution, you can pass subplots() the resolu- 
tion using the dpi parameter: 


fig, ax = plt.subplots(figsize=(10, 6), dpi=128) 


This should help make the most efficient use of the space available on 
your screen. 


TRY IT YOURSELF 


15-3. Molecular Motion: Modify rw_visual.py by replacing ax.scatter() with 
ax.plot(). To simulate the path of a pollen grain on the surface of a drop of 


water, pass in the rw.x_values and rw.y values, and include a linewidth argu- 
ment. Use 5,000 instead of 50,000 points to keep the plot from being too busy. 


15-4. Modified Random Walks: In the RandomWalk class, x step and y step are 
generated from the same set of conditions. The direction is chosen randomly 
from the list [1, -1] and the distance from the list [0, 1, 2, 3, 4]. Modify the 
values in these lists to see what happens to the overall shape of your walks. Try 
a longer list of choices for the distance, such as O through 8, or remove the -1 
from the x or y-direction list. 

15-5. Refactoring: The fill walk() method is lengthy. Create a new method 
called get step() to determine the direction and distance for each step, and then 
calculate the step. You should end up with two calls to get step() in fill walk(): 


x step - self.get step() 
y step - self.get step() 


This refactoring should reduce the size of fill walk() and make the 
method easier to read and understand. 


Rolling Dice with Plotly 


In this section, we'll use Plotly to produce interactive visualizations. Plotly is 
particularly useful when you're creating visualizations that will be displayed 
in a browser, because the visualizations will scale automatically to fit the 
viewer's screen. These visualizations are also interactive; when the user hov- 
ers over certain elements on the screen, information about those elements 
is highlighted. We'll build our initial visualization in just a couple lines of 
code using Plotly Express, a subset of Plotly that focuses on generating plots 
with as little code as possible. Once we know our plot is correct, we'll cus- 
tomize the output just as we did with Matplotlib. 

In this project, we'll analyze the results of rolling dice. When you roll 
one regular, six-sided die, you have an equal chance of rolling any of the 
numbers from 1 through 6. However, when you use two dice, you're more 
likely to roll certain numbers than others. We'll try to determine which 
numbers are most likely to occur by generating a dataset that represents 
rolling dice. Then we'll plot the results of a large number of rolls to deter- 
mine which results are more likely than others. 

This work helps model games involving dice, but the core ideas also apply 
to games that involve chance of any kind, such as card games. It also relates to 
many real-world situations where randomness plays a significant factor. 
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Installing Plotly 
Install Plotly using pip, just as you did for Matplotlib: 


$ python -m pip install --user plotly 
$ python -m pip install --user pandas 


Plotly Express depends on pandas, which is a library for working effi- 
ciently with data, so we need to install that as well. If you used python3 or 
something else when installing Matplotlib, make sure you use the same 
command here. 

To see what kind of visualizations are possible with Plotly, visit the gal- 
lery of chart types at Attps://plotly.com/python. Each example includes source 
code, so you can see how Plotly generates the visualizations. 


Creating the Die Class 


We'll create the following Die class to simulate the roll of one die: 


die.py from random import randint 


class Die: 
"""A class representing a single die.""" 
e def init (self, num sides-6): 
"""Assume a six-sided die. 
self.num sides - num sides 


def roll(self): 
""""Return a random value between 1 and number of sides. 
e return randint(1, self.num sides) 


The init () method takes one optional argument 6.. With the Die 
class, when an instance of our die is created, the number of sides will be six 
if no argument is included. If an argument is included, that value will set 
the number of sides on the die. (Dice are named for their number of sides: 
a six-sided die is a D6, an eight-sided die is a D8, and so on.) 

The roll() method uses the randint() function to return a random number 
between 1 and the number of sides 9. This function can return the starting 
value (1), the ending value (num sides), or any integer between the two. 


Rolling the Die 


Before creating a visualization based on the Die class, let's roll a D6, print 
the results, and check that the results look reasonable: 


die visual py ^ from die import Die 


it Create a D6. 
@ die = Die() 
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# Make some rolls, and store results in a list. 
results = [] 


6 for roll num in range(100): 


result - die.roll() 
results.append(result) 


print(results) 


We create an instance of Die with the default six sides €. Then we 
roll the die 100 times and store the result of each roll in the list results. 
Here's a sample set of results: 


[4, 6, 5, 6, 1, 5, 6, 3, 5, 3, 5, 3, 2, 2, 1, 3, 1, 5, 3, 6, 3, 6, 5, 4, 
1,1,4,2, 3, 6, 4, 2, 6, 4, 1, 3, 2, 5, 6, 3, 6, 2, 1, 1, 3, 4, 1, 4, 
3, 5, 1, 4, 5, 5, 2, 3, 3, 1, 2, 3, 5, 6, 2, 5, 6, 1, 3, 2, 1, 1, 1, 6, 
5, 5, 2, 2, 6, 4, 1, 4, 5, 1, 1, 1, 4, 5, 3, 3, 1, 3, 5, 4, 5, 6, 5, 4, 


1, 5, 1, 2] 


A quick scan of these results shows that the Die class seems to be work- 
ing. We see the values 1 and 6, so we know the smallest and largest possible 
values are being returned, and because we don’t see 0 or 7, we know all 
the results are in the appropriate range. We also see each number from 1 
through 6, which indicates that all possible outcomes are represented. Let's 
determine exactly how many times each number appears. 


Analyzing the Results 


We'll analyze the results of rolling one D6 by counting how many times we 
roll each number: 


9 for roll num in range(1000): 


# Analyze the results. 
frequencies = [] 


O poss results = range(1, die.num_sides+1) 


for value in poss results: 
frequency = results.count(value) 
frequencies. append(frequency) 


print (frequencies) 


Because we’re no longer printing the results, we can increase the num- 
ber of simulated rolls to 1000 6. To analyze the rolls, we create the empty 
list frequencies to store the number of times each value is rolled. We then 
generate all the possible results we could get; in this example, that's all the 
numbers from 1 to however many sides die has @. We loop through the pos- 
sible values, count how many times each number appears in results 6, and 
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then append this value to frequencies O. We print this list before making a 
visualization: 


[155, 167, 168, 170, 159, 181] 


These results look reasonable: we see six frequencies, one for each possi- 
ble number when you roll a D6. We also see that no frequency is significantly 
higher than any other. Now let’s visualize these results. 


Making a Histogram 


Now that we have the data we want, we can generate a visualization in just a 
couple lines of code using Plotly Express: 


import plotly.express as px 


# Visualize the results. 
fig = px.bar(x=poss results, y=frequencies) 
fig. show() 


We first import the plotly.express module, using the conventional alias 
px. We then use the px.bar() function to create a bar graph. In the simplest 
use of this function, we only need to pass a set of x-values and a set of y-values. 
Here the x-values are the possible results from rolling a single die, and the 
y-values are the frequencies for each possible result. 

The final line calls fig.show(), which tells Plotly to render the resulting 
chart as an HTML file and open that file in a new browser tab. The result is 
shown in Figure 15-12. 

This is a really simple chart, and it’s certainly not complete. But this 
is exactly how Plotly Express is meant to be used; you write a couple lines 
of code, look at the plot, and make sure it represents the data the way you 
want it to. If you like what you see, you can move on to customizing ele- 
ments of the chart such as labels and styles. But if you want to explore other 
possible chart types, you can do so now, without having spent extra time on 
customization work. Feel free to try this now by changing px.bar() to some- 
thing like px.scatter() or px.line(). You can find a full list of available chart 
types at hitps://plotly.com/python/plotly-express. 

This chart is dynamic and interactive. If you change the size of your 
browser window, the chart will resize to match the available space. If you 
hover over any of the bars, you'll see a pop-up highlighting the specific data 
related to that bar. 
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Figure 15-12: The initial plot produced by Plotly Express 


Customizing the Plot 


Now that we know we have the correct kind of plot and our data is being 
represented accurately, we can focus on adding the appropriate labels and 
styles for the chart. 

The first way to customize a plot with Plotly is to use some optional 
parameters in the initial call that generates the plot, in this case, px.bar(). 
Here's how to add an overall title and a label for each axis: 


--snip-- 
# Visualize the results. 


@ title = "Results of Rolling One D6 1,000 Times" 
@ labels = ('x': 'Result', 'y': 'Frequency of Result'} 


fig = px.bar(x=poss results, y-frequencies, title-title, labels-labels) 
fig.show() 


We first define the title that we want, here assigned to title 6. To 
define axis labels, we write a dictionary 6. The keys in the dictionary refer 
to the labels we want to customize, and the values are the custom labels we 
want to use. Here we give the x-axis the label Result and the y-axis the label 
Frequency of Result. The call to px.bar() now includes the optional argu- 
ments title and labels. 

Now when the plot is generated it includes an appropriate title and a 
label for each axis, as shown in Figure 15-18. 
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Figure 15-13: A simple bar chart created with Plotly 


Rolling Two Dice 


Rolling two dice results in larger numbers and a different distribution of 

results. Let's modify our code to create two D6 dice to simulate the way we 

roll a pair of dice. Each time we roll the pair, we'll add the two numbers (one 
from each die) and store the sum in results. Save a copy of die visual.py as 

dice visual.py and make the following changes: 


dice visual py ^ import plotly.express as px 
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from die import Die 


# Create two D6 dice. 
die 1 - Die() 
die 2 - Die() 


# Make some rolls, and store results in a list. 
results - [] 
for roll num in range(1000): 
o result = die 1.roll() + die 2.roll() 
results.append(result) 


# Analyze the results. 
frequencies - [] 
O max result = die 1.num sides + die 2.num sides 
© poss results = range(2, max_result+1) 
for value in poss results: 
frequency = results.count(value) 
frequencies.append(frequency) 


Chapter 15 


# Visualize the results. 

title = "Results of Rolling Two D6 Dice 1,000 Times" 

labels = ('x': 'Result', 'y': ‘Frequency of Result'} 

fig = px.bar(x=poss results, y=frequencies, title=title, labels=labels) 
fig.show() 


After creating two instances of Die, we roll the dice and calculate the sum 
of the two dice for each roll 6. The smallest possible result (2) is the sum of 
the smallest number on each die. The largest possible result (12) is the sum 
of the largest number on each die, which we assign to max result @. The 
variable max result makes the code for generating poss results much easier 
to read 6. We could have written range(2, 13), but this would work only 
for two D6 dice. When modeling real-world situations, it's best to write code 
that can easily model a variety of situations. This code allows us to simu- 
late rolling a pair of dice with any number of sides. 

After running this code, you should see a chart that looks like Figure 15-14. 


eee (D € 122001 | © +6 
DELI 


& 5 
Results of Rolling Two D6 Dice 1,000 Times 


c— 

120 

100 

80 

60 

40 

a H 
9 2 4 6 8 10 12 


Result 


Frequency of Result 


Figure 15-14: Simulated results of rolling two six-sided dice 1,000 times 


This graph shows the approximate distribution of results you’re likely to 
get when you roll a pair of D6 dice. As you can see, you're least likely to roll 
a 2 ora 12 and most likely to roll a 7. This happens because there are six 
ways to roll a 7: 1 and 6, 2 and 5, 3 and 4, 4 and 3, 5 and 2, and 6 and 1. 


Further Customizations 


There's one issue that we should address with the plot we just generated. 
Now that there are 11 bars, the default layout settings for the x-axis leave 
some of the bars unlabeled. While the default settings work well for most 
visualizations, this chart would look better with all of the bars labeled. 
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Plotly has an update_layout() method that can be used to make a wide 
variety of updates to a figure after it’s been created. Here’s how to tell Plotly 
to give each bar its own label: 


dice visual.py 


dice visual 
_déd10.py 
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# Further customize chart. 
fig.update layout(xaxis dtick-1) 


The update layout() method acts on the fig object, which represents the 
overall chart. Here we use the xaxis dtick argument, which specifies the 
distance between tick marks on the x-axis. We set that spacing to 1, so that 
every bar is labeled. When you run dice visual.py again, you should see a 
label on each bar. 


Rolling Dice of Different Sizes 


Let's create a six-sided die and a ten-sided die, and see what happens when 
we roll them 50,000 times: 


# Create a D6 and a D10. 


@ die 2 = Die(10) 
for roll num in range(50 000): 


+ 


@ title = "Results of Rolling a D6 and a D10 50,000 Times" 


To make a D10, we pass the argument 10 when creating the second Die 
instance ® and change the first loop to simulate 50,000 rolls instead of 
1,000. We change the title of the graph as well 6. 


Figure 15-15 shows the resulting chart. Instead of one most likely result, 
there are five such results. This happens because there’s still only one way 
to roll the smallest value (1 and 1) and the largest value (6 and 10), but the 
smaller die limits the number of ways you can generate the middle numbers. 
There are six ways to roll a 7, 8, 9, 10, or 11, these are the most common 
results, and you're equally likely to roll any one of them. 
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Figure 15-15: The results of rolling a six-sided die and a ten-sided die 50,000 times 
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Our ability to use Plotly to model the rolling of dice gives us considerable 
freedom in exploring this phenomenon. In just minutes, you can simulate a 
tremendous number of rolls using a large variety of dice. 


Saving Figures 


When you have a figure you like, you can always save the chart as an HTML 
file through your browser. But you can also do so programmatically. To save 
your chart as an HTML file, replace the call to fig.show() with a call to fig 
.write html(): 


fig.write html('dice visual d6d10.html') 


The write html() method requires one argument: the name of the file 
to write to. If you only provide a filename, the file will be saved in the same 
directory as the .py file. You can also call write html() with a Path object, and 
write the output file anywhere you want on your system. 
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TRY IT YOURSELF 


15-6. Two D8s: Create a simulation showing what happens when you roll two 
eight-sided dice 1,000 times. Try to picture what you think the visualization will 
look like before you run the simulation, then see if your intuition was correct. 
Gradually increase the number of rolls until you start to see the limits of your 
system’s capabilities. 

15-7. Three Dice: When you roll three Dé dice, the smallest number you can roll 
is 3 and the largest number is 18. Create a visualization that shows what hap- 
pens when you roll three Dé dice. 


15-8. Multiplication: When you roll two dice, you usually add the two numbers 
together to get the result. Create a visualization that shows what happens if you 
multiply these numbers by each other instead. 


15-9. Die Comprehensions: For clarity, the listings in this section use the long 


form of for loops. If you're comfortable using list comprehensions, try writing a 


comprehension for one or both of the loops in each of these programs. 

15-10. Practicing with Both Libraries: Try using Matplotlib to make a die-rolling 
visualization, and use Plotly to make the visualization for a random walk. (You'll 
need to consult the documentation for each library to complete this exercise.) 


Summary 
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In this chapter, you learned to generate datasets and create visualizations 
of that data. You created simple plots with Matplotlib and used a scatter 
plot to explore random walks. You also created a histogram with Plotly, and 
used it to explore the results of rolling dice of different sizes. 

Generating your own datasets with code is an interesting and power- 
ful way to model and explore a wide variety of real-world situations. As you 
continue to work through the data visualization projects that follow, keep 
an eye out for situations you might be able to model with code. Look at the 
visualizations you see in news media, and see if you can identify those that 
were generated using methods similar to the ones you're learning in these 
projects. 

In Chapter 16, you'll download data from online sources and continue 
to use Matplotlib and Plotly to explore that data. 


DOWNLOADING DATA 


In this chapter, you'll download datasets 
from online sources and create working 
visualizations of that data. You can find 
an incredible variety of data online, much of 
which hasn't been examined thoroughly. The ability 
to analyze this data allows you to discover patterns 
and connections that no one else has found. 


We'll access and visualize data stored in two common data formats: 
CSV and JSON. We'll use Python's csv module to process weather data 
stored in the CSV format and analyze high and low temperatures over time 
in two different locations. We'll then use Matplotlib to generate a chart 
based on our downloaded data to display variations in temperature in two 
dissimilar environments: Sitka, Alaska, and Death Valley, California. Later 
in the chapter, we'll use the json module to access earthquake data stored 
in the GeoJSON format and use Plotly to draw a world map showing the 
locations and magnitudes of recent earthquakes. 


By the end of this chapter, you'll be prepared to work with various types 
of datasets in different formats, and you'll have a deeper understanding of 
how to build complex visualizations. Being able to access and visualize 
online data is essential to working with a wide variety of real-world datasets. 


The CSV File Format 


One simple way to store data in a text file is to write the data as a series of 
values separated by commas, called comma-separated values. The resulting files 
are CSV files. For example, here's a chunk of weather data in CSV format: 


"USWO0025333" , "SITKA AIRPORT, AK US","2021-01-01", , "44", "40" 


This is an excerpt of weather data from January 1, 2021, in Sitka, Alaska. 
Itincludes the day's high and low temperatures, as well as a number of other 
measurements from that day. CSV files can be tedious for humans to read, 
but programs can process and extract information from them quickly and 
accurately. 

We'll begin with a small set of CSV-formatted weather data recorded 
in Sitka; it is available in this book's resources at hitps://ehmatthes.github.io/ 
pcc .3e. Make a folder called weather data inside the folder where you're sav- 
ing this chapter's programs. Copy the file sitka weather 07-2021. simple.csu 
into this new folder. (After you download this book's resources, you'll have 
all the files you need for this project.) 


The weather data in this project was originally downloaded from https://ncdc 
.noaa.gov/cdo-web. 


Parsing the CSV File Headers 


Python's csv module in the standard library parses the lines in a CSV file 
and allows us to quickly extract the values we're interested in. Let's start by 
examining the first line of the file, which contains a series of headers for 
the data. These headers tell us what kind of information the data holds: 


sitka highs.py from pathlib import Path 
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import csv 


6 path = Path('weather data/sitka weather 07-2021 simple.csv') 
lines - path.read text().splitlines() 


6 reader = csv.reader(lines) 
© header row = next(reader) 
print(header row) 


We first import Path and the csv module. We then build a Path object 
that looks in the weather data folder, and points to the specific weather data 
file we want to work with 6€. We read the file and chain the splitlines() 
method to get a list of all lines in the file, which we assign to lines. 
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sitka highs.py 


Next, we build a reader object 0. This is an object that can be used 
to parse each line in the file. To make a reader object, call the function 
csv.reader() and pass it the list of lines from the CSV file. 

When given a reader object, the next() function returns the next line 
in the file, starting from the beginning of the file. Here we call next() only 
once, so we get the first line of the file, which contains the file headers 8. 
We assign the data that's returned to header row. As you can see, header row 
contains meaningful, weather-related headers that tell us what information 
each line of data holds: 


['STATION', 'NAME', 'DATE', 'TAVG', 'TMAX', 'TMIN'] 


The reader object processes the first line of comma-separated values in 
the file and stores each value as an item in a list. The header STATION repre- 
sents the code for the weather station that recorded this data. The position 
of this header tells us that the first value in each line will be the weather 
station code. The NAME header indicates that the second value in each line 
is the name of the weather station that made the recording. The rest of 
the headers specify what kinds of information were recorded in each read- 
ing. The data we're most interested in for now are the date (DATE), the high 
temperature (TMAX), and the low temperature (TMIN). This is a simple dataset 
that contains only temperature-related data. When you download your own 
weather data, you can choose to include a number of other measurements 
relating to wind speed, wind direction, and precipitation data. 


Printing the Headers and Their Positions 


To make it easier to understand the file header data, let's print each header 
and its position in the list: 


for index, column header in enumerate(header row): 
print(index, column header) 


The enumerate() function returns both the index of each item and the 
value of each item as you loop through a list. (Note that we've removed the 
line print(header row) in favor of this more detailed version.) 

Here's the output showing the index of each header: 


O STATION 
1 NAME 
2 DATE 
3 TAVG 
4 TMAX 
5 TMIN 
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We can see that the dates and their high temperatures are stored in col- 
umns 2 and 4. To explore this data, we'll process each row of data in sitka 
weather 07-2021 .simple.csu and extract the values with the indexes 2 and 4. 


Extracting and Reading Data 


Now that we know which columns of data we need, let's read in some of that 
data. First, we'll read in the high temperature for each day: 


sitka highs.py 


# Extract high temperatures. 
9 highs = [] 
@ for row in reader: 
e high = int(row[4]) 
highs .append(high) 


print (highs) 


We make an empty list called highs € and then loop through the 
remaining rows in the file 6. The reader object continues from where it left 
off in the CSV file and automatically returns each line following its current 
position. Because we've already read the header row, the loop will begin 
at the second line where the actual data begins. On each pass through the 
loop we pull the data from index 4, corresponding to the header TMAX, and 
assign it to the variable high 6. We use the int() function to convert the 
data, which is stored as a string, to a numerical format so we can use it. We 
then append this value to highs. 

The following listing shows the data now stored in highs: 


[61, 60, 66, 60, 65, 59, 58, 58, 57, 60, 60, 60, 57, 58, 60, 61, 63, 63, 70, 
64, 59, 63, 61, 58, 59, 64, 62, 70, 70, 73, 66] 


We've extracted the high temperature for each date and stored each 
value in a list. Now let's create a visualization of this data. 


Plotting Data in a Temperature Chart 


To visualize the temperature data we have, we'll first create a simple plot of 
the daily highs using Matplotlib, as shown here: 


sitka highs.py 


import matplotlib.pyplot as plt 
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# Plot the high temperatures. 
plt.style.use('seaborn') 
fig, ax = plt.subplots() 

6 ax.plot(highs, color-'red') 


# Format plot. 
O ax.set title("Daily High Temperatures, July 2021", fontsize=24) 
© ax.set xlabel('', fontsize-16) 

ax.set ylabel("Temperature (F)", fontsize-16) 

ax.tick params(labelsize-16) 


plt.show() 


We pass the list of highs to plot() and pass color-'red' to plot the points 
in red 6. (We'll plot the highs in red and the lows in blue.) We then specify 
a few other formatting details, such as the title, font size, and labels 9, just 
as we did in Chapter 15. Because we have yet to add the dates, we won't 
label the x-axis, but ax.set xlabel() does modify the font size to make 
the default labels more readable 6. Figure 16-1 shows the resulting plot: a 
simple line graph of the high temperatures for July 2021 in Sitka, Alaska. 
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Figure 16-1: A line graph showing daily high temperatures for July 2021 
in Sitka, Alaska 


The datetime Module 


Let’s add dates to our graph to make it more useful. The first date from the 
weather data file is in the second row of the file: 


"USWO0025333" , " SITKA AIRPORT, AK US","2021-07-01", ,"61","53" 


The data will be read in as a string, so we need a way to convert the 
string "2021-07-01" to an object representing this date. We can construct 
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an object representing July 1, 2021, using the strptime() method from the 
datetime module. Let's see how strptime() works in a terminal session: 


»»» from datetime import datetime 

>>> first date = datetime.strptime('2021-07-01', '%Y-%m-%d' ) 
>>> print(first_date) 

2021-07-01 00:00:00 


We first import the datetime class from the datetime module. Then we 
call the method strptime() with the string containing the date we want to 
process as its first argument. The second argument tells Python how the 
date is formatted. In this example, '%Y-' tells Python to look for a four-digit 
year before the first dash; '%m-' indicates a two-digit month before the sec- 
ond dash; and '%d' means the last part of the string is the day of the month, 
from 1 to 31. 

The strptime() method can take a variety of arguments to determine 
how to interpret the date. Table 16-1 shows some of these arguments. 


Table 16-1: Date and Time Formatting Arguments from 
the datetime Module 


Argument Meaning 


XA Weekday name, such as Monday 
AB Month name, such as January 

%m Month, as a number (01 to 12) 

%d Day of the month, as a number (01 to 31) 
XY Four-digit year, such as 2019 

Ay Two-digit year, such as 19 

%H Hour, in 24-hour format (00 to 23) 
XI Hour, in 12-hour format (01 to 12) 
Ap AM or PM 

4M Minutes (00 to 59) 

4S Seconds (00 to 61) 

Plotting Dates 


We can improve our plot by extracting dates for the daily high temperature 
readings, and using these dates on the x-axis: 


from datetime import datetime 


reader = csv.reader(lines) 
header_row = next(reader) 


# Extract dates and high temperatures. 
@ dates, highs = [], [] 
for row in reader: 
e current date = datetime.strptime(row[2], '%Y-%m-%d' ) 
high = int(row[4]) 
dates.append(current date) 
highs.append(high) 


# Plot the high temperatures. 
plt.style.use('seaborn') 
fig, ax = plt.subplots() 

© ax.plot(dates, highs, color-'red') 


# Format plot. 
ax.set_title("Daily High Temperatures, July 2021", fontsize=24) 
ax.set_xlabel('', fontsize=16) 
O fig.autofmt xdate() 
ax.set ylabel("Temperature (F)", fontsize=16) 
ax.tick params(labelsize-16) 


plt.show() 


We create two empty lists to store the dates and high temperatures 
from the file €. We then convert the data containing the date information 
(row[2]) to a datetime object € and append it to dates. We pass the dates and 
the high temperature values to plot() 6. The call to fig.autofmt xdate() O 
draws the date labels diagonally to prevent them from overlapping. Figure 16-2 
shows the improved graph. 
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Figure 16-2: The graph is more meaningful, now that it has dates on the x-axis. 
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Plotting a Longer Timeframe 


With our graph set up, let's include additional data to get a more complete 
picture of the weather in Sitka. Copy the file sitka, weather 2021 simple.csv, 
which contains a full year's worth of weather data for Sitka, to the folder 
where you're storing the data for this chapter's programs. 

Now we can generate a graph for the entire year's weather: 


--snip-- 
path - Path('weather data/sitka weather 2021 simple.csv') 
lines = path.read text().splitlines() 


# Format plot. 
ax.set_title("Daily High Temperatures, 2021", fontsize=24) 
ax.set_xlabel('', fontsize=16) 


We modify the filename to use the new data file sttka_weather_2021 
—simple.csv, and we update the title of our plot to reflect the change in its 
content. Figure 16-3 shows the resulting plot. 
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Figure 16-3: A year's worth of data 


Plotting a Second Data Series 


We can make our graph even more useful by including the low tempera- 
tures. We need to extract the low temperatures from the data file and then 
add them to our graph, as shown here: 


--snip-- 
reader - csv.reader(lines) 
header row - next(reader) 


# Extract dates, and high and low temperatures. 
dates, highs, lows = [], [], [] 


for row in reader: 
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current date = datetime.strptime(row[2], '%Y-%m-%d'" ) 
high = int(row[4]) 
e low - int(row[5]) 
dates.append(current date) 
highs.append(high) 
lows.append(low) 


# Plot the high and low temperatures. 
plt.style.use('seaborn') 
fig, ax = plt.subplots() 
ax.plot(dates, highs, color-'red') 

© ax.plot(dates, lows, color-'blue') 


# Format plot. 
@ ax.set_title("Daily High and Low Temperatures, 2021", fontsize=24) 
--snip-- 


We add the empty list lows to hold low temperatures 6, and then we 
extract and store the low temperature for each date from the sixth position 
in each row (row[5]) 9. We add a call to plot() for the low temperatures and 
color these values blue 6. Finally, we update the title O. Figure 16-4 shows 
the resulting chart. 
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Figure 16-4: Two data series on the same plot 


Shading an Area in the Chart 


Having added two data series, we can now examine the range of tempera- 
tures for each day. Let's add a finishing touch to the graph by using shading 
to show the range between each day's high and low temperatures. To do so, 
we'll use the fill between() method, which takes a series of x-values and two 
series of y-values and fills the space between the two series of y-values: 


--snip-- 
# Plot the high and low temperatures. 
plt.style.use('seaborn') 
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e ax.plot(dates, highs, color-'red', alpha-0.5) 


ax.plot(dates, lows, color-'blue', alpha-0.5) 


@ ax.fill between(dates, highs, lows, facecolor-'blue', alpha-0.1) 


The alpha argument controls a color's transparency 6.. An alpha value 
of 0 is completely transparent, and a value of 1 (the default) is completely 
opaque. By setting alpha to 0.5, we make the red and blue plot lines appear 
lighter. 

We pass fill between() the list dates for the x-values and then the two 
y-value series highs and lows @. The facecolor argument determines the 
color of the shaded region; we give it a low alpha value of 0.1 so the filled 
region connects the two data series without distracting from the informa- 
tion they represent. Figure 16-5 shows the plot with the shaded region 
between the highs and lows. 
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Figure 16-5: The region between the two datasets is shaded. 


The shading helps make the range between the two datasets immedi- 
ately apparent. 


Error Checking 


We should be able to run the sitka, highs lows.py code using data for any 
location. But some weather stations collect different data than others, and 
some occasionally malfunction and fail to collect some of the data they're 
supposed to. Missing data can result in exceptions that crash our programs, 
unless we handle them properly. 

For example, let's see what happens when we attempt to generate a tem- 
perature plot for Death Valley, California. Copy the file death valley 2021 
_simple.csv to the folder where you're storing the data for this chapter's 
programs. 


First, let’s run the code to see the headers that are included in this 
data file: 


death_valley from pathlib import Path 
_highs_lows.py import csv 


path = Path('weather data/death valley 2021 simple.csv') 
lines - path.read text().splitlines() 


reader - csv.reader(lines) 
header row - next(reader) 


for index, column header in enumerate(header row): 
print(index, column header) 


Here's the output: 


O STATION 
1 NAME 
2 DATE 
3 TMAX 
4 TMIN 
5 TOBS 


The date is in the same position, at index 2. But the high and low tem- 
peratures are at indexes 3 and 4, so we'll need to change the indexes in our 
code to reflect these new positions. Instead of including an average temper- 
ature reading for the day, this station includes TOBS, a reading for a specific 
observation time. 

Change sitka highs lows.py to generate a graph for Death Valley using 
the indexes we just noted, and see what happens: 


death valley 
highs lows.py path = Path('weather data/death valley 2021 simple.csv') 


high - eod. 
low = int(row[4]) 


We update the program to read from the Death Valley data file, and we 
change the indexes to correspond to this file’s TMAX and TMIN positions. 
When we run the program, we get an error: 


Traceback (most recent call last): 

File "death valley highs lows.py", line 17, in <module> 
high = int(row[3]) 

@ ValueError: invalid literal for int() with base 10: 
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The traceback tells us that Python can’t process the high temperature for 
one of the dates because it can't turn an empty string (‘') into an integer @. 
Rather than looking through the data to find out which reading is missing, 
we'll just handle cases of missing data directly. 

We'll run error-checking code when the values are being read from the 
CSV file to handle exceptions that might arise. Here's how to do this: 


e try: 
except ValueError: 

print(f"Missing data for {current_date}") 
else: 


© © 


© title = "Daily High and Low Temperatures, 2021\nDeath Valley, CA" 
ax.set_title(title, fontsize=20) 


Each time we examine a row, we try to extract the date and the high 
and low temperature @. If any data is missing, Python will raise a ValueError 
and we handle it by printing an error message that includes the date of the 
missing data @. After printing the error, the loop will continue processing 
the next row. If all data for a date is retrieved without error, the else block 
will run and the data will be appended to the appropriate lists 6. Because 
we're plotting information for a new location, we update the title to include 
the location on the plot, and we use a smaller font size to accommodate the 
longer title O. 

When you run death, valley highs lows.py now, you'll see that only one 
date had missing data: 


Missing data for 2021-05-04 00:00:00 


Because the error is handled appropriately, our code is able to generate a 
plot, which skips over the missing data. Figure 16-6 shows the resulting plot. 

Comparing this graph to the Sitka graph, we can see that Death Valley 
is warmer overall than southeast Alaska, as we expect. Also, the range of 
temperatures each day is greater in the desert. The height of the shaded 
region makes this clear. 
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Figure 16-6: Daily high and low temperatures for Death Valley 


Many datasets you work with will have missing, improperly formatted, 
or incorrect data. You can use the tools you learned in the first half of this 
book to handle these situations. Here we used a try-except-else block to 
handle missing data. Sometimes you'll use continue to skip over some data, 
or use remove() or del to eliminate some data after it’s been extracted. Use 
any approach that works, as long as the result is a meaningful, accurate 
visualization. 


Downloading Your Own Data 


To download your own weather data, follow these steps: 


l. Visit the NOAA Climate Data Online site at https://www.ncdc.noaa.gov/ 
cdo-web. In the Discover Data By section, click Search Tool. In the Select 
a Dataset box, choose Daily Summaries. 

2. Select a date range, and in the Search For section, choose ZIP Codes. 
Enter the ZIP code you're interested in and click Search. 

3. On the next page, you'll see a map and some information about the 
area you're focusing on. Below the location name, click View Full 
Details, or click the map and then click Full Details. 

4. Scroll down and click Station List to see the weather stations that are 
available in this area. Click one of the station names and then click 
Add to Cart. This data is free, even though the site uses a shopping cart 
icon. In the upper-right corner, click the cart. 

5. In Select the Output Format, choose Custom GHCN-Daily CSV. Make 
sure the date range is correct and click Continue. 


Downloading Data 341 


6. On the next page, you can select the kinds of data you want. You can 
download one kind of data (for example, focusing on air temperature) 
or you can download all the data available from this station. Make your 
choices and then click Continue. 


7. On the last page, you'll see a summary of your order. Enter your email 
address and click Submit Order. You'll receive a confirmation that your 
order was received, and in a few minutes, you should receive another 
email with a link to download your data. 


The data you download should be structured just like the data we 
worked with in this section. It might have different headers than those you 
saw in this section, but if you follow the same steps we used here, you should 
be able to generate visualizations of the data you're interested in. 


TRY IT YOURSELF 


16-1. Sitka Rainfall: Sitka is located in a temperate rainforest, so it gets a fair 
amount of rainfall. In the data file sitka weather. 2021. full.csv is a header called 
PRCP, which represents daily rainfall amounts. Make a visualization focusing on 
the data in this column. You can repeat the exercise for Death Valley if you're 
curious how little rainfall occurs in a desert. 


16-2. Sitka-Death Valley Comparison: The temperature scales on the Sitka and 
Death Valley graphs reflect the different data ranges. To accurately compare 
the temperature range in Sitka to that of Death Valley, you need identical scales 
on the y-axis. Change the settings for the y-axis on one or both of the charts in 
Figures 16-5 and 16-6. Then make a direct comparison between temperature 
ranges in Sitka and Death Valley (or any two places you want to compare]. 


16-3. San Francisco: Are temperatures in San Francisco more like tempera- 
tures in Sitka or temperatures in Death Valley? Download some data for San 
Francisco, and generate a high-low temperature plot for San Francisco to make 
a comparison. 


16-4. Automatic Indexes: In this section, we hardcoded the indexes correspond- 
ing to the TMIN and TMAX columns. Use the header row to determine the indexes 


for these values, so your program can work for Sitka or Death Valley. Use the sta- 


tion name to automatically generate an appropriate title for your graph as well. 


16-5. Explore: Generate a few more visualizations that examine any other 
weather aspect you're interested in for any locations you're curious about. 


Mapping Global Datasets: GeoJSON Format 


In this section, you'll download a dataset representing all the earthquakes 
that have occurred in the world during the previous month. Then you'll 
make a map showing the location of these earthquakes and how significant 
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each one was. Because the data is stored in the GeoJSON format, we’ll work 
with it using the json module. Using Plotly’s scatter_geo() plot, you’ll create 
visualizations that clearly show the global distribution of earthquakes. 


Downloading Earthquake Data 


Make a folder called eq data inside the folder where you're saving this 
chapter's programs. Copy the file eq 1 day ml.geojson into this new folder. 
Earthquakes are categorized by their magnitude on the Richter scale. This 
file includes data for all earthquakes with a magnitude M1 or greater that 
took place in the last 24 hours (at the time of this writing). This data comes 
from one of the United States Geological Survey's earthquake data feeds, at 
https://earthquake.usgs.gou/earthquakes/feed. 


Examining GeoJSON Data 


When you open eq 1 day ml.geojson, you'll see that it's very dense and hard 
to read: 


(" type" :"FeatureCollection", "metadata": ("generated":1649052296000,... 


(" type" : "Feature", "properties":("mag":1.6, "place":"63 km SE of Ped... 
(" type" : "Feature", "properties":("mag" :2.2, "place":"27 km SSE of Ca... 
(" type" : "Feature" ,"properties":("mag" :3.7, "place":"102 km SSE of S... 
(" type" : "Feature" ,"properties":("mag" :2.92000008, "place":"49 km SE... 


(" type" : "Feature", "properties" :("mag" :1.4, "place":"44 km NE of Sus... 
--snip-- 


This file is formatted more for machines than humans. But we can see 
that the file contains some dictionaries, as well as information that we're 
interested in, such as earthquake magnitudes and locations. 

The json module provides a variety of tools for exploring and working 
with JSON data. Some of these tools will help us reformat the file so we can 
look at the raw data more easily before we work with it programmatically. 

Let's start by loading the data and displaying it in a format that's easier 
to read. This is a long data file, so instead of printing it, we'll rewrite the 
data to a new file. Then we can open that file and scroll back and forth 
through the data more easily: 


from pathlib import Path 
import json 


# Read data as a string and convert to a Python object. 
path = Path('eq data/eq data 1 day m1.geojson') 
contents - path.read text() 

@ all eq data = json.loads(contents) 


# Create a more readable version of the data file. 

@ path = Path('eq data/readable eq data.geojson') 

© readable contents = json.dumps(all eq data, indent=4) 
path.write text(readable contents) 
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We read the data file as a string, and use json. loads() to convert the 
string representation of the file to a Python object @. This is the same 
approach we used in Chapter 10. In this case, the entire dataset is converted 
to a single dictionary, which we assign to all eq data. We then define a new 
path where we can write this same data in a more readable format . The 
json.dumps() function that you saw in Chapter 10 can take an optional indent 
argument 6, which tells it how much to indent nested elements in the data 
structure. 

When you look in your eq. data directory and open the file readable eq 
. data.json, here's the first part of what you'll see: 


{ 


"type": "FeatureCollection", 
"metadata": { 
"generated": 1649052296000, 
"url": "https://earthquake.usgs.gov/earthquakes/.../1.0 day.geojson", 
"title": "USGS Magnitude 1.0+ Earthquakes, Past Day", 
"status": 200, 
"api": "1.10.3", 
"count": 160 
b 


"features": [ 


The first part of the file includes a section with the key "metadata" ®. 
This tells us when the data file was generated and where we can find the 
data online. It also gives us a human-readable title and the number of 
earthquakes included in this file. In this 24-hour period, 160 earthquakes 
were recorded. 

This GeoJSON file has a structure that’s helpful for location-based data. 
The information is stored in a list associated with the key "features" @. 
Because this file contains earthquake data, the data is in list form where 
every item in the list corresponds to a single earthquake. This structure 
might look confusing, but it’s quite powerful. It allows geologists to store as 
much information as they need to in a dictionary about each earthquake, 
and then stuff all those dictionaries into one big list. 

Let’s look at a dictionary representing a single earthquake: 


"type": "Feature", 
"properties": { 


"mag": 1.6, 

--snip-- 

"title": "M 1.6 - 27 km NNW of Susitna, Alaska" 
b 
"geometry": { 


"type": "Point", 

"coordinates": [ 
-150.7585, 
61.7591, 
56.3 
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] 
b 
"id": "ako224bju1jx" 
b 


The key "properties" contains a lot of information about each earth- 
quake 6. We're mainly interested in the magnitude of each earthquake, 
associated with the key "mag". We're also interested in the "title" of each 
event, which provides a nice summary of its magnitude and location 6. 

The key "geometry" helps us understand where the earthquake occurred 6. 
We'll need this information to map each event. We can find the longitude @ 
and the latitude 9 for each earthquake in a list associated with the key 
"coordinates". 

This file contains way more nesting than we'd use in the code we write, 
so if it looks confusing, don't worry: Python will handle most of the com- 
plexity. We'll only be working with one or two nesting levels at a time. We'll 
start by pulling out a dictionary for each earthquake that was recorded in 
the 24-hour time period. 


When we talk about locations, we often say the location's latitude first, followed by its 
longitude. This convention probably arose because humans discovered latitude long 
before we developed the concept of longitude. However, many geospatial frameworks 
list the longitude first and then the latitude, because this corresponds to the (x, y) 
convention we use in mathematical representations. The GeoJSON format follows the 
(longitude, latitude) convention. If you use a different framework, it’s important to 
learn what convention that framework follows. 


Making a List of All Earthquakes 


First, we'll make a list that contains all the information about every earth- 
quake that occurred. 


# Examine all earthquakes in the dataset. 
all eq dicts = all eq data['features'] 
print(len(all eq dicts)) 


We take the data associated with the key 'features' in the all eq data 
dictionary, and assign it to all eq dicts. We know this file contains records 
of 160 earthquakes, and the output verifies that we've captured all the 
earthquakes in the file: 


160 
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Notice how short this code is. The neatly formatted file readable_eq_data 
json has over 6,000 lines. But in just a few lines, we can read through all 
that data and store it in a Python list. Next, we'll pull the magnitudes from 
each earthquake. 


Extracting Magnitudes 


We can loop through the list containing data about each earthquake, and 
extract any information we want. Let's pull out the magnitude of each 
earthquake: 


eq explore 
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9 mags = [] 
for eq dict in all eq dicts: 
e mag = eq dict['properties']['mag'] 
mags .append(mag) 


print (mags[:10]) 


We make an empty list to store the magnitudes, and then loop through 
the list all eq dicts ®. Inside this loop, each earthquake is represented by the 
dictionary eq dict. Each earthquake's magnitude is stored in the 'properties' 
section of this dictionary, under the key 'mag' 6. We store each magnitude in 
the variable mag and then append it to the list mags. 

We print the first 10 magnitudes, so we can see whether we're getting 
the correct data: 


[1.6, 1.6, 2.2, 3.7, 2.92000008, 1.4, 4.6, 4.5, 1.9, 1.8] 


Next, we'll pull the location data for each earthquake, and then we can 
make a map of the earthquakes. 


Extracting Location Data 


The location data for each earthquake is stored under the key "geometry". 
Inside the geometry dictionary is a "coordinates" key, and the first two values 
in this list are the longitude and latitude. Here's how we'll pull this data: 
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mags, lons, lats = [], [], [] 
e lon - eq dict['geometry' ][' coordinates" ][0] 
lat = eq dict['geometry']['coordinates'][1] 


lons.append(lon) 
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lats.append(lat) 


srintil Si). 
print(lats[:5]) 


We make empty lists for the longitudes and latitudes. The code 
eq dict['geometry'] accesses the dictionary representing the geometry ele- 
ment of the earthquake 6. The second key, 'coordinates', pulls the list 
of values associated with 'coordinates'. Finally, the 0 index asks for the first 
value in the list of coordinates, which corresponds to an earthquake's 
longitude. 

When we print the first 5 longitudes and latitudes, the output shows 
that we're pulling the correct data: 


[-150.7585, -153.4716, -148.7531, -159.6267, -155.248336791992] 
[61.7591, 59.3152, 63.1633, 54.5612, 18.7551670074463] 


With this data, we can move on to mapping each earthquake. 


Building a World Map 


Using the information we've pulled so far, we can build a simple world map. 
Although it won't look presentable yet, we want to make sure the informa- 
tion is displayed correctly before focusing on style and presentation issues. 
Here's the initial map: 


import plotly.express as px 


title = 'Global Earthquakes ' 


@ fig = px.scatter geo(lat-lats, lon-lons, title=title) 


fig.show() 


We import plotly.express with the alias px, just as we did in Chapter 15. 
The scatter geo() function 6 allows you to overlay a scatterplot of geo- 
graphic data on a map. In the simplest use of this chart type, you only need 
to provide a list of latitudes and a list of longitudes. We pass the list lats to 
the lat argument, and lons to the lon argument. 

When you run this file, you should see a map that looks like the one in 
Figure 16-7. This again shows the power of the Plotly Express library; in just 
three lines of code, we have a map of global earthquake activity. 
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Figure 16-7: A simple map showing where all the earthquakes in the last 24 hours 
occurred 


Now that we know the information in our dataset is being plotted cor- 
rectly, we can make a few changes to make the map more meaningful and 
easier to read. 


Representing Magnitudes 


A map of earthquake activity should show the magnitude of each earth- 
quake. We can also include more data, now that we know the data is being 
plotted correctly. 


# Read data as a string and convert to a Python o 
path = Path('eq data/eq data 30 day m1.geojson') 
contents - path.read text() 


)D]Ject. 


title = ‘Global Earthquakes 
fig = px.scatter geo(lat-lats, lon-lons, size-mags, title=title) 


We load the file eq data. 30 day ml.geojson, to include a full 30 days’ worth 
of earthquake activity. We also use the size argument in the px.scatter geo() 
call, which specifies how the points on the map will be sized. We pass the list 
mags to size, so earthquakes with a higher magnitude will show up as larger 
points on the map. 

The resulting map is shown in Figure 16-8. Earthquakes usually 
occur near tectonic plate boundaries, and the longer period of earth- 
quake activity included in this map reveals the exact locations of these 
boundaries. 
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Figure 16-8: The map now shows the magnitude of all earthquakes in the last 30 days. 


This map is better, but it’s still difficult to pick out which points repre- 
sent the most significant earthquakes. We can improve this further by using 
color to represent magnitudes as well. 


Customizing Marker Colors 


We can use Plotly’s color scales to customize each marker’s color, according 
to the severity of the corresponding earthquake. We'll also use a different 
projection for the base map. 


--snip-- 

fig - px.scatter geo(lat-lats, lon-lons, size-mags, title-title, 
color-mags, 
color continuous scale='Viridis', 
labels={'color':'Magnitude'}, 
projection='natural earth’, 


All the significant changes here occur in the px.scatter_geo() function 
call. The color argument tells Plotly what values it should use to determine 
where each marker falls on the color scale 60. We use the mags list to deter- 
mine the color for each point, just as we did with the size argument. 

The color continuous scale argument tells Plotly which color scale to 
use @. Viridisis a color scale that ranges from dark blue to bright yellow, 
and it works well for this dataset. By default, the color scale on the right of 
the map is labeled color; this is not representative of what the colors actually 
mean. The labels argument, shown in Chapter 15, takes a dictionary as a 
value ©. We only need to set one custom label on this chart, making sure 
the color scale is labeled Magnitude instead of color. 
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We add one more argument, to modify the base map over which the 
earthquakes are plotted. The projection argument accepts a number of com- 
mon map projections O. Here we use the ‘natural earth’ projection, which 
rounds the ends of the map. Also, note the trailing comma after this last 
argument. When a function call has a long list of arguments spanning mul- 
tiple lines like this, it’s common practice to add a trailing comma so you’re 
always ready to add another argument on the next line. 

When you run the program now, you'll see a much nicer-looking map. 
In Figure 16-9, the color scale shows the severity of individual earthquakes; 
the most severe earthquakes stand out as light-yellow points, in contrast to 
many darker points. You can also tell which regions of the world have more 
significant earthquake activity. 
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Global Earthquakes 
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Figure 16-9: In 30 days’ worth of earthquakes, color and size are used to represent 
the magnitude of each earthquake. 


Other Color Scales 


You can choose from a number of other color scales. To see the available 
color scales, enter the following two lines in a Python terminal session: 


»»» import plotly.express as px 
»»» px.colors.named colorscales() 
['aggrnyl', 'agsunset', 'blackbody', ..., 'mygbm'] 


Feel free to try out these color scales in the earthquake map, or with any 
dataset where continuously varying colors can help show patterns in the data. 


Adding Hover Text 


To finish this map, we'll add some informative text that appears when you 

hover over the marker representing an earthquake. In addition to showing 
the longitude and latitude, which appear by default, we'll show the magni- 
tude and provide a description of the approximate location as well. 


To make this change, we need to pull a little more data from the file: 


eq world | --snip-- 

.map.py 9 mags, lons, lats, eq titles = 
mag = eq dict['properties 
lon = eq dict['geometry'] 
lat = eq dict['geometry'] 

e eq title - eq dict['prop 
mags .append(mag) 
lons.append(lon) 
lats.append(lat) 
eq titles.append(eq title) 


L E (1, 0] 
['mag'] 
['coordinates'][0] 
['coordinates'][1] 
rties']['title'] 


[ 
: 


title - 'Global Earthquakes' 
fig = px.scatter geo(lat-lats, lon-lons, size-mags, title-title, 
--snip-- 
projection-'natural earth', 
e hover name-eq titles, 


) 
fig.show() 


We first make a list called eq titles to store the title of each earth- 
quake 9. The 'title' section of the data contains a descriptive name of 
the magnitude and location of each earthquake, in addition to its longi- 
tude and latitude. We pull this information and assign it to the variable 
eq title @, and then append it to the list eq titles. 

In the px.scatter geo() call, we pass eq titles to the hover name argu- 
ment 6. Plotly will now add the information from the title of each earthquake 
to the hover text on each point. When you run this program, you should be 
able to hover over any marker, see a description of where that earthquake took 
place, and read its exact magnitude. An example of this information is shown 
in Figure 16-10. 


r * 


Figure 16-10: The hover text now includes a summary of each earthquake. 


This is impressive! In less than 30 lines of code, we've created a visu- 
ally appealing and meaningful map of global earthquake activity that also 
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illustrates the geological structure of the planet. Plotly offers a wide range 
of ways you can customize the appearance and behavior of your visualiza- 
tions. Using Plotly’s many options, you can make charts and maps that show 
exactly what you want them to. 


TRY IT YOURSELF 


16-6. Refactoring: The loop that pulls data from a11 eq dicts uses variables for 
the magnitude, longitude, latitude, and title of each earthquake before append- 
ing these values to their appropriate lists. This approach was chosen for clarity 
in how to pull data from a GeoJSON file, but it’s not necessary in your code. 
Instead of using these temporary variables, pull each value from eq_dict and 
append it to the appropriate list in one line. Doing so should shorten the body 
of this loop to just four lines. 


16-7. Automated Title: In this section, we used the generic title Global 
Earthquakes. Instead, you can use the title for the dataset in the metadata part 
of the GeoJSON file. Pull this value and assign it to the variable title. 


16-8. Recent Earthquakes: You can find online data files containing information 
about the most recent earthquakes over 1-hour, 1-day, 7-day, and 30-day peri- 
ods. Go to https://earthquake.usgs.gov/earthquakes/feed/v1.0/geojson.php and 
you'll see a list of links to datasets for various time periods, focusing on earth- 
quakes of different magnitudes. Download one of these datasets and create a 


visualization of the most recent earthquake activity. 


16-9. World Fires: In the resources for this chapter, you'll find a file called 
world fires 1 day.csv. This file contains information about fires burning in differ- 
ent locations around the globe, including the latitude, longitude, and brightness 
of each fire. Using the data-processing work from the first part of this chapter 
and the mapping work from this section, make a map that shows which parts of 
the world are affected by fires. 

You can download more recent versions of this data at hitps://earthdata 
.nasa.gov/earth-observation-data/near-real-time/firms/active-fire-data. You can 
find links to the data in CSV format in the SHP, KML, and TXT Files section. 


Summary 


Chapter 16 


In this chapter, you learned how to work with real-world datasets. You pro- 
cessed CSV and GeoJSON files, and extracted the data you want to focus 
on. Using historical weather data, you learned more about working with 
Matplotlib, including how to use the datetime module and how to plot multi- 
ple data series on one chart. You plotted geographical data on a world map 
in Plotly, and learned to customize the style of the map. 

As you gain experience working with CSV and JSON files, you'll be able 
to process almost any data you want to analyze. You can download most 
online datasets in either or both of these formats. By working with these 


formats, you'll be able to learn how to work with other data formats more 
easily as well. 

In the next chapter, you'll write programs that automatically gather 
their own data from online sources, and then you'll create visualizations 
of that data. These are fun skills to have if you want to program as a hobby 
and are critical skills if you're interested in programming professionally. 
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WORKING WITH APIS 


In this chapter, you'll learn how to write 
a self-contained program that generates a 
visualization based on data it retrieves. Your 

program will use an application programming 
interface (API) to automatically request specific infor- 
mation from a website and then use that information 


to generate a visualization. Because programs written 


like this will always use current data to generate a visualization, even 
when that data might be rapidly changing, the visualization will always 
be up to date. 


Using an API 


An API is a part of a website designed to interact with programs. Those 
programs use very specific URLs to request certain information. This kind 
of request is called an API call. The requested data will be returned in an 
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easily processed format, such as JSON or CSV. Most apps that use external 
data sources, such as apps that integrate with social media sites, rely on 
API calls. 


Git and GitHub 


We'll base our visualization on information from GitHub (hitps://github.com), 
asite that allows programmers to collaborate on coding projects. We'll use 
GitHub's API to request information about Python projects on the site, and 
then generate an interactive visualization of the relative popularity of these 
projects using Plotly. 

GitHub takes its name from Git, a distributed version control system. 
Git helps people manage their work on a project in a way that prevents 
changes made by one person from interfering with changes other people 
are making. When you implement a new feature in a project, Git tracks the 
changes you make to each file. When your new code works, you commit 
the changes you've made, and Git records the new state of your project. If 
you make a mistake and want to revert your changes, you can easily return 
to any previously working state. (To learn more about version control using 
Git, see Appendix D.) Projects on GitHub are stored in repositories, which 
contain everything associated with the project: its code, information on its 
collaborators, any issues or bug reports, and so on. 

When users on GitHub like a project, they can "star" it to show their 
support and keep track of projects they might want to use. In this chapter, 
we'll write a program to automatically download information about the 
most-starred Python projects on GitHub, and then we'll create an informa- 
tive visualization of these projects. 


Requesting Data Using an API Call 


GitHub's API lets you request a wide range of information through API 
calls. To see what an API call looks like, enter the following into your brows- 
er's address bar and press ENTER: 


https://api.github.com/search/repositories?q=language: python+sort: stars 


This call returns the number of Python projects currently hosted on 
GitHub, as well as information about the most popular Python repositories. 
Let’s examine the call. The first part, https://api.github.com/, directs the 
request to the part of GitHub that responds to API calls. The next part, 
search/repositories, tells the API to conduct a search through all the reposi- 
tories on GitHub. 

The question mark after repositories signals that we’re about to pass 
an argument. The q stands for query, and the equal sign (=) lets us begin 
specifying a query (q=). By using language: python, we indicate that we want 
information only on repositories that have Python as the primary language. 
The final part, +sort:stars, sorts the projects by the number of stars they’ve 
been given. 


python 
_repos.py 


e 


The following snippet shows the first few lines of the response: 


"total count": 8961993, 
"incomplete results": true, 
"items": [ 

{ 


"id": 54346799, 

"node id": "MDEwO1J1cG9zaXRvcnk1NDMONjc500==", 
"name": "public-apis", 

"full name": "public-apis/public-apis", 
--snip-- 


You can see from the response that this URL is not primarily intended 
to be entered by humans, because it's in a format that's meant to be pro- 
cessed by a program. GitHub found just under nine million Python projects 
as of this writing €.. The value for "incomplete results" is true, which tells us 
that GitHub didn't fully process the query 9. GitHub limits how long each 
query can run, in order to keep the API responsive for all users. In this case 
it found some of the most popular Python repositories, but it didn't have 
time to find all of them; we'll fix that in a moment. The "items" returned 
are displayed in the list that follows, which contains details about the most 
popular Python projects on GitHub 6. 


Installing Requests 


The Requests package allows a Python program to easily request information 
from a website and examine the response. Use pip to install Requests: 


$ python -m pip install --user requests 


If you use a command other than python to run programs or start a ter- 
minal session, such as python3, your command will look like this: 


$ python3 -m pip install --user requests 


Processing an API Response 


Now we'll write a program to automatically issue an API call and process 
the results: 


import requests 


# Make an API call and check the response. 


6 url = "https://api.github.com/search/repositories" 


url += "?q=language:python+sort:stars+stars :>10000" 


@ headers = {"Accept": "application/vnd.github.v3+json"} 
© r = requests.get(url, headers-headers) 
© print(f'Status code: {r.status code}") 
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# Convert the response object to a dictionary. 
© response dict = r.json() 


# Process results. 
print (response dict.keys()) 


We first import the requests module. Then we assign the URL of the 
API call to the url variable €. This is a long URL, so we break it into two 
lines. The first line is the main part of the URL, and the second line is 
the query string. We’ve included one more condition to the original query 
string: stars:>10000, which tells GitHub to only look for Python repositories 
that have more than 10,000 stars. This should allow GitHub to return a 
complete, consistent set of results. 

GitHub is currently on the third version of its API, so we define headers 
for the API call that ask explicitly to use this version of the API, and return 
the results in the JSON format @. Then we use requests to make the call 
to the API 6. We call get() and pass it the URL and the header that we 
defined, and we assign the response object to the variable r. 

The response object has an attribute called status_code, which tells us 
whether the request was successful. (A status code of 200 indicates a success- 
ful response.) We print the value of status_code so we can make sure the call 
went through successfully O. We asked the API to return the information in 
JSON format, so we use the json() method to convert the information to a 
Python dictionary 8. We assign the resulting dictionary to response dict. 

Finally, we print the keys from response dict and see the following 
output: 


Status code: 200 
dict keys(['total count', 'incomplete results', 'items']) 


Because the status code is 200, we know that the request was successful. 
The response dictionary contains only three keys: 'total count', 'incomplete 
_results', and 'items'. Let's take a look inside the response dictionary. 


Working with the Response Dictionary 


With the information from the API call represented as a dictionary, we 
can work with the data stored there. Let's generate some output that sum- 
marizes the information. This is a good way to make sure we received the 
information we expected, and to start examining the information we're 
interested in: 


@ print(f'Total repositories: {response dict['total_count']}") 
print(f'Complete results: {not response dict['incomplete_results']}") 


# Explore information about the repositories. 
O repo dicts = response dict['items'] 
print(f"Repositories returned: {len(repo dicts)}") 


# Examine the first repository. 
© repo dict = repo dicts[0] 
© print(f"\nKeys: {len(repo dict)}") 
© for key in sorted(repo dict.keys()): 
print(key) 


We start exploring the response dictionary by printing the value asso- 
ciated with 'total count', which represents the total number of Python 
repositories returned by this API call 6. We also use the value associated 
with 'incomplete results', so we'll know if GitHub was able to fully process 
the query. Rather than printing this value directly, we print its opposite: a 
value of True will indicate that we received a complete set of results. 

The value associated with 'items' is a list containing a number of dic- 
tionaries, each of which contains data about an individual Python reposi- 
tory. We assign this list of dictionaries to repo dicts @. We then print the 
length of repo dicts to see how many repositories we have information for. 

To look closer at the information returned about each repository, we 
pull out the first item from repo dicts and assign it to repo dict 6. We then 
print the number of keys in the dictionary to see how much information we 
have 9. Finally, we print all the dictionary's keys to see what kind of infor- 
mation is included 9. 

The results give us a clearer picture of the actual data: 


Status code: 200 
@ Total repositories: 248 
© Complete results: True 
Repositories returned: 30 


© Keys: 78 
allow forking 
archive url 
archived 
--snip-- 
url 
visiblity 
watchers 
watchers count 


At the time of this writing, there are only 248 Python repositories with 
over 10,000 stars €. We can see that GitHub was able to fully process the 
API call @. In this response, GitHub returned information about the first 30 
repositories that match the conditions of our query. If we want more reposi- 
tories, we can request additional pages of data. 
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GitHub’s API returns a lot of information about each repository: there 
are 78 keys in repo dict ©. When you look through these keys, you'll get 
a sense of the kind of information you can extract about a project. (The 
only way to know what information is available through an API is to read 
the documentation or to examine the information through code, as we're 
doing here.) 

Let's pull out the values for some of the keys in repo dict: 


print("\nSelected information about first repository:") 

© print(f'Name: (repo dict['name']}") 

© print(f'Owner: (repo dict['owner']['login']]") 

© print(f'Stars: (repo dict['stargazers count']}") 
print(f"Repository: (repo dict['html url']]") 

© print(f'Created: {repo dict['created at']j") 

© print(f'Updated: [repo dict['updated at']j") 
print(f'Description: {repo dict['description']]") 


Here, we print the values for a number of keys from the first repository's 
dictionary. We start with the name of the project 0. An entire dictionary 
represents the project's owner, so we use the key owner to access the diction- 
ary representing the owner, and then use the key login to get the owner's 
login name @. Next, we print how many stars the project has earned ® and 
the URL for the project's GitHub repository. We then show when it was cre- 
ated O and when it was last updated 9. Finally, we print the repository's 
description. 

The output should look something like this: 


Status code: 200 

Total repositories: 248 
Complete results: True 
Repositories returned: 30 


Selected information about first repository: 

Name: public-apis 

Owner: public-apis 

Stars: 191493 

Repository: https://github.com/public-apis/public-apis 
Created: 2016-03-20T23:49:42Z 

Updated: 2022-05-12T06:37:11Z 

Description: A collective list of free APIs 


We can see that the most-starred Python project on GitHub as of this 
writing is public-apis. Its owner is an organization with the same name, and 
it has been starred by almost 200,000 GitHub users. We can see the URL 
for the project's repository, its creation date of March 2016, and that it was 
updated recently. Additionally, the description tells us that public-apis con- 
tains a list of free APIs that programmers might be interested in. 


python 
_repos.py 


Summarizing the Top Repositories 


When we make a visualization for this data, we’ll want to include more 
than one repository. Let’s write a loop to print selected information about 
each repository the API call returns so we can include them all in the 
visualization: 


@ print("\nSelected information about each repository:") 
@ for repo dict in repo dicts: 


We first print an introductory message 6. Then we loop through all 
the dictionaries in repo_dicts @. Inside the loop, we print the name of each 
project, its owner, how many stars it has, its URL on GitHub, and the proj- 
ect’s description: 


Status code: 200 

Total repositories: 248 
Complete results: True 
Repositories returned: 30 


Selected information about each repository: 


Name: public-apis 

Owner: public-apis 

Stars: 191494 

Repository: https://github.com/public-apis/public-apis 
Description: A collective list of free APIs 


Name: system-design-primer 

Owner: donnemartin 

Stars: 179952 

Repository: https://github.com/donnemartin/system-design-primer 

Description: Learn how to design large-scale systems. Prep for the system 
design interview. Includes Anki flashcards. 

--snip-- 


Name: PayloadsAllTheThings 

Owner: swisskyrepo 

Stars: 37227 

Repository: https://github.com/swisskyrepo/PayloadsAllTheThings 

Description: A list of useful payloads and bypass for Web Application Security 
and Pentest/CTF 


Working with APIs 361 


362 


Se) 


Some interesting projects appear in these results, and it might be worth 
looking at a few. But don’t spend too much time here, because we’re about 
to create a visualization that will make the results much easier to read. 


Monitoring API Rate Limits 


Most APIs have rate limits, which means there’s a limit to how many requests 
you can make in a certain amount of time. To see if you’re approaching 
GitHub's limits, enter https://api.github.com/rate_limit into a web browser. You 
should see a response that begins like this: 


{ 

"resources": { 
--snip-- 
"search": { 

"limit": 10, 


"remaining": 9, 

"reset": 1652338832, 

"used": 1, 

"resource": "search" 
h 


--snip-- 


The information we’re interested in is the rate limit for the search 
API @. We see that the limit is 10 requests per minute @ and that we have 
9 requests remaining for the current minute ©. The value associated with 
the key "reset" represents the time in Unix or epoch time (the number of 
seconds since midnight on January 1, 1970) when our quota will reset O. If 
you reach your quota, you'll get a short response that lets you know you've 
reached the API limit. If you reach the limit, just wait until your quota 
resets. 


Many APIs require you to register and obtain an API key or access token to make API 
calls. As of this writing, GitHub has no such requirement, but if you obtain an access 
token, your limits will be much higher. 


Visualizing Repositories Using Plotly 
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Let’s make a visualization using the data we’ve gathered to show the rela- 
tive popularity of Python projects on GitHub. We'll make an interactive bar 
chart: the height of each bar will represent the number of stars the project 
has acquired, and you'll be able to click the bar’s label to go to that project's 
home on GitHub. 


Save a copy of the program we’ve been working on as python_repos 
_visual.py, then modify it so it reads as follows: 


python repos import requests 
_visualpy ^ import plotly.express as px 


# Process overall results. 


# Process repository information. 


epo dicts = response dict 


e repo names, stars - []; [] 
for repo dict in repo dicts: 
repo names.append(repo dict['name']) 
stars.append(repo dict['stargazers count']) 


# Make visualization. 
O fig = px.bar(x-repo names, y=stars) 
fig.show() 


We import Plotly Express and then make the API call as we have been 
doing. We continue to print the status of the API call response so we'll 
know if there is a problem 6. When we process the overall results, we 
continue to print the message confirming that we got a complete set of 
results 9. We remove the rest of the print() calls because we're no longer in 
the exploratory phase; we know we have the data we want. 

We then create two empty lists © to store the data we'll include in the 
initial chart. We'll need the name of each project to label the bars (repo names) 
and the number of stars to determine the height of the bars (stars). In the 
loop, we append the name of each project and the number of stars it has to 
these lists. 

We make the initial visualization with just two lines of code O. This is 
consistent with Plotly Express's philosophy that you should be able to see 
your visualization as quickly as possible before refining its appearance. 
Here we use the px.bar() function to create a bar chart. We pass the list 
repo names as the x argument and stars as the y argument. 

Figure 17-1 shows the resulting chart. We can see that the first few projects 
are significantly more popular than the rest, but all of them are important 
projects in the Python ecosystem. 
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Figure 17-1: The moststarred Python projects on GitHub 


Styling the Chart 


Plotly supports a number of ways to style and customize the plots, once you 
know the information in the plot is correct. We'll make some changes in 
the initial px.bar() call and then make some further adjustments to the fig 
object after it's been created. 

We'll start styling the chart by adding a title and labels for each axis: 


python repos — --snip-- 
_visualpy | 5 Make visualization. 
title - "Most-Starred Python Projects on GitHub" 


labels = ('x': 'Repository', 'y': 'Stars'] 
fig = px.bar(x-repo names, y-stars, title-title, labels-labels) 


€ fig.update layout(title font size-28, xaxis title font size-20, 
yaxis title font size-20) 


fig.show() 


We first add a title and labels for each axis, as we did in Chapters 15 and 
16. We then use the fig.update layout() method to modify specific elements 
of the chart 6. Plotly uses a convention where aspects of a chart element are 
connected by underscores. As you become familiar with Plotly's documen- 
tation, you'll start to see consistent patterns in how different elements of a 
chart are named and modified. Here we set the title font size to 28 and the 
font size for each axis title to 20. The result is shown in Figure 17-2. 
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Figure 17-2: A title has been added to the main chart, and to each axis as well. 


Adding Custom Tooltips 


In Plotly, you can hover the cursor over an individual bar to show the infor- 

mation the bar represents. This is commonly called a tooltip, and in this 

case, it currently shows the number of stars a project has. Let's create a cus- 

tom tooltip to show each project's description as well as the project's owner. 
We need to pull some additional data to generate the tooltips: 


--snip-- 
# Process repository information. 
repo dicts - response dict['items'] 


6 repo names, stars, hover texts = [], [], [] 


for repo dict in repo dicts: 
repo names.append(repo dict['name']) 
stars.append(repo dict['stargazers count']) 


# Build hover texts. 

owner - repo dict['owner']['login'] 
description = repo dict['description'] 
hover text = f"{owner}<br />{description}" 
hover_texts.append(hover_text) 


# Make visualization. 
title = "Most-Starred Python Projects on GitHub" 
labels = {'x': ‘Repository’, 'y': 'Stars'] 


© fig = px.bar(x-repo names, y-stars, title-title, labels=labels, 


hover_name=hover_texts) 
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fig.update layout(title font size-28, xaxis title font size-20, 
yaxis title font size-20) 


fig.show() 


We first define a new empty list, hover texts, to hold the text we want to 
display for each project 0. In the loop where we process the data, we pull 
the owner and the description for each project 6. Plotly allows you to use 
HTML code within text elements, so we generate a string for the label with 
a line break («br /») between the project owner's username and the descrip- 
tion 6. We then append this label to the list hover texts. 

In the px.bar() call, we add the hover name argument and pass it hover 
texts O. This is the same approach we used to customize the label for each 
dot in the map of global earthquake activity. As Plotly creates each bar, it 
will pull labels from this list and only display them when the viewer hovers 
over a bar. Figure 17-3 shows one of these custom tooltips. 


django 
The Web framework for perfectionists with deadlines. 


Repository=django 
Stars=64.409k 


Figure 17-3: Hovering over a bar shows the project’s owner and description. 


Adding Clickable Links 


Because Plotly allows you to use HTML on text elements, we can easily add 
links to a chart. Let’s use the x-axis labels as a way to let the viewer visit any 
project’s home page on GitHub. We need to pull the URLs from the data 
and use them when generating the x-axis labels: 


--snip-- 
# Process repository information. 
repo dicts - response dict['items'] 
@ repo links, stars, hover texts = [], [], [] 
for repo dict in repo dicts: 
# Turn repo names into active links. 
repo name - repo dict['name'] 
e repo url - repo dict['html url'] 


e repo link = f"<a href='{repo_url}'>{repo_name}</a>" 
repo links.append(repo link) 


We update the name of the list we're creating from repo names to repo 
.links to more accurately communicate the kind of information we're put- 
ting together for the chart 6. We then pull the URL for the project from 
repo dict and assign it to the temporary variable repo url @. Next, we gen- 
erate a link to the project ©. We use the HTML anchor tag, which has the 
form «a href='URL'>link text</a>, to generate the link. We then append this 
link to repo links. 

When we call px.bar(), we use repo links for the x-values in the chart. 
The result looks the same as before, but now the viewer can click any of the 
project names at the bottom of the chart to visit that project's home page 
on GitHub. Now we have an interactive, informative visualization of data 
retrieved through an API! 


Customizing Marker Colors 


Once a chart has been created, almost any aspect of the chart can be cus- 
tomized through an update method. We've used the update layout() method 
previously. Another method, update traces(), can be used to customize the 
data that's represented on a chart. 

Let's change the bars to a darker blue, with some transparency: 


fig.update traces(marker color-'SteelBlue', marker opacity-0.6) 


In Plotly, a trace refers to a collection of data on a chart. The update 
_traces() method can take a number of different arguments; any argument 
that starts with marker affects the markers on the chart. Here we set each 
marker's color to 'SteelBlue'; any named CSS color will work here. We also set 
the opacity of each marker to 0.6. An opacity of 1.0 will be entirely opaque, 
and an opacity of 0 will be entirely invisible. 
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More About Plotly and the GitHub API 


Plotly’s documentation is extensive and well organized; however, it can be 
hard to know where to start reading. A good place to start is with the article 
“Plotly Express in Python,” at hitps://plotly.com/python/plotly-express. This is an 
overview of all the plots you can make with Plotly Express, and you can find 
links to longer articles about each individual chart type. 

If you want to understand how to customize Plotly charts better, the 
article “Styling Plotly Express Figures in Python" will expand on what you've 
seen in Chapters 15-17. You can find this article at hitps://plotly.com/python/ 
styling-plotly-express. 

For more about the GitHub API, refer to its documentation at hitps:// 
docs.github.com/en/rest. Here you'll learn how to pull a wide variety of infor- 
mation from GitHub. To expand on what you saw in this project, look for 
the Search section of the reference in the sidebar. If you have a GitHub 
account, you can work with your own data as well as the publicly available 
data from other users' repositories. 


The Hacker News API 


To explore how to use API calls on other sites, let's take a quick look at 
Hacker News (hitps://news.ycombinator.com). On Hacker News, people share 
articles about programming and technology and engage in lively discus- 
sions about those articles. The Hacker News API provides access to data 
about all submissions and comments on the site, and you can use the API 
without having to register for a key. 

The following call returns information about the current top article as 
of this writing: 


https: //hacker-news. firebaseio.com/v0/item/31353677.json 


When you enter this URL in a browser, you'll see that the text on the 
page is enclosed by braces, meaning it's a dictionary. But the response is 
difficult to examine without some better formatting. Let's run this URL 
through the json.dumps() method, like we did in the earthquake project 
in Chapter 16, so we can explore the kind of information that's returned 
about an article: 


hn_article.py import requests 
import json 


# Make an API call, and store the response. 

url = "https://hacker-news.firebaseio.com/vo/item/31353677.json" 
r - requests.get(url) 

print(f'Status code: {r.status_code}") 


# Explore the structure of the data. 

response dict - r.json() 

response string - json.dumps(response dict, indent-4) 
© print(response string) 
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hn 


_submissions.py 


Everything in this program should look familiar, because we’ve used 
it all in the previous two chapters. The main difference here is that we can 
print the formatted response string @ instead of writing it to a file, because 
the output is not particularly long. 

The output is a dictionary of information about the article with the ID 
31353677: 

t 

"by": "sohkamyung", 

"descendants": 302, 

"id": 31353677, 

"kids": [ 

31354987, 
31354235, 
--snip-- 


] 


3 

core": 785, 

"time": 1652361401, 

"title": "Astronomers reveal first image of the black hole 
at the heart of our galaxy", 

"type": "story", 

"url": "https://public.nrao.edu/news/.../" 


The dictionary contains a number of keys we can work with. The key 
"descendants" tells us the number of comments the article has received 6. 
The key "kids" provides the IDs of all comments made directly in response 
to this submission @. Each of these comments might have comments of 
their own as well, so the number of descendants a submission has is usually 
greater than its number of kids. We can see the title of the article being dis- 
cussed ® and a URL for the article being discussed as well O. 

The following URL returns a simple list of all the IDs of the current top 
articles on Hacker News: 


https://hacker-news.firebaseio.com/vO/topstories.json 


We can use this call to find out which articles are on the home page 
right now, and then generate a series of API calls similar to the one we just 
examined. With this approach, we can print a summary of all the articles 
on the front page of Hacker News at the moment: 


from operator import itemgetter 
import requests 


# Make an API call and check the response. 


@ url = "https: //hacker-news.firebaseio.com/vO/topstories.json" 


r = requests.get(url) 
print(f'Status code: {r.status_code}") 


# Process information about each submission. 


6 submission ids = r.json() 
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© submission dicts = [] 
for submission id in submission ids[:5]: 
# Make a new API call for each submission. 
o url = f"https://hacker-news.firebaseio.com/vo/item/(submission id).json" 
r = requests.get(url) 
print(f'id: (submission id}\tstatus: (r.status code)") 
response dict - r.json() 


# Build a dictionary for each article. 
e submission dict = ( 
'title': response dict['title'], 
'hn link': f"https://news.ycombinator.com/item?id={submission id)", 
'comments': response dict['descendants'], 


L5) submission dicts.append(submission dict) 


@ submission dicts = sorted(submission dicts, key-itemgetter('comments'), 
reverse=True) 


© for submission dict in submission dicts: 
print(f"\nTitle: (submission dict['title']}") 
print(f"Discussion link: {submission dict['hn link'])") 
print(f"Comments: (submission dict['comments'])") 


First, we make an API call and print the status of the response 6. This 
API call returns a list containing the IDs of up to 500 of the most popular 
articles on Hacker News at the time the call is issued. We then convert the 
response object to a Python list @, which we assign to submission ids. We'll 
use these IDs to build a set of dictionaries, each of which contains informa- 
tion about one of the current submissions. 

We set up an empty list called submission dicts to store these dictionar- 
ies 6. We then loop through the IDs of the top 30 submissions. We make 
a new API call for each submission by generating a URL that includes the 
current value of submission id O. We print the status of each request along 
with its ID, so we can see whether it's successful. 

Next, we create a dictionary for the submission currently being pro- 
cessed ©. We store the title of the submission, a link to the discussion page 
for that item, and the number of comments the article has received so far. 
Then we append each submission dict to the list submission dicts ©. 

Each submission on Hacker News is ranked according to an overall 
score based on a number of factors, including how many times it's been 
voted on, how many comments it's received, and how recent the submis- 
sion is. We want to sort the list of dictionaries by the number of comments. 
To do this, we use a function called itemgetter() @, which comes from the 
operator module. We pass this function the key 'comments', and it pulls the 
value associated with that key from each dictionary in the list. The sorted() 
function then uses this value as its basis for sorting the list. We sort the list 
in reverse order, to place the most-commented stories first. 

Once the list is sorted, we loop through the list 9 and print out three 
pieces of information about each of the top submissions: the title, a link 


to the discussion page, and the number of comments the submission cur- 
rently has: 


Status code: 200 

id: 31390506 status: 200 
id: 31389893 status: 200 
id: 31390742 status: 200 
--snip-- 


Title: Fly.io: The reclaimer of Heroku's magic 
Discussion link: https://news.ycombinator.com/item?id-31390506 
Comments: 134 


Title: The weird Hewlett Packard FreeDOS option 
Discussion link: https://news.ycombinator.com/item?id-31389893 
Comments: 64 


Title: Modern JavaScript Tutorial 

Discussion link: https://news.ycombinator.com/item?id-31390742 
Comments: 20 

--snip-- 


You would use a similar process to access and analyze information with 
any API. With this data, you could make a visualization showing which 
submissions have inspired the most active recent discussions. This is also 
the basis for apps that provide a customized reading experience for sites 
like Hacker News. To learn more about what kind of information you can 
access through the Hacker News API, visit the documentation page at 
https://github.com/HacherNews/A PI. 


Hacker News sometimes allows companies it supports to make special hiring posts, 
and comments are disabled on these posts. If you run this program while one of these 
posts is present, you'll get a KeyError. If this causes an issue, you cam wrap the code 
that builds submission dict im a try-except block and skip over these posts. 


TRY IT YOURSELF 


17-1. Other Languages: Modify the API call in python repos.py so it generates 


a chart showing the most popular projects in other languages. Try languages 
such as JavaScript, Ruby, C, Java, Perl, Haskell, and Go. 


17-2. Active Discussions: Using the data from hn submissions.py, make a bar 
chart showing the most active discussions currently happening on Hacker News. 
The height of each bar should correspond to the number of comments each sub- 
mission has. The label for each bar should include the submission's title and act 
as a link to the discussion page for that submission. If you get a KeyError when 
creating a chart, use a try-except block to skip over the promotional posts. 
[continued] 
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17-3. Testing python_repos.py: In python_repos.py, we printed the value of 
status_code to make sure the API call was successful. Write a program called 
test_python_repos.py that uses pytest to assert that the value of status code 

is 200. Figure out some other assertions you can make: for example, that the 
number of items returned is expected and that the total number of repositories is 
greater than a certain amount. 


17-4. Further Exploration: Visit the documentation for Plotly and either the 
GitHub API or the Hacker News API. Use some of the information you find there 
to either customize the style of the plots we've already made or pull some dif- 
ferent information and create your own visualizations. If you're curious about 
exploring other APIs, take a look at the APIs mentioned in the GitHub repository 
at https;//github.com/public-apis. 


Summary 


In this chapter, you learned how to use APIs to write self-contained pro- 
grams that automatically gather the data they need and use that data to 
create a visualization. You used the GitHub API to explore the most-starred 
Python projects on GitHub, and you also looked briefly at the Hacker News 
API. You learned how to use the Requests package to automatically issue 
an API call and how to process the results of that call. We also introduced 
some Plotly settings that further customize the appearance of the charts 
you generate. 

In the next chapter, you'll use Django to build a web application as your 
final project. 
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GETTING STARTED WITH DJANGO 


As the internet has evolved, the line between 
websites and mobile apps has blurred. 
Websites and apps both help users interact 
with data in a variety of ways. Fortunately, you 
can use Django to build a single project that serves a 
dynamic website as well as a set of mobile apps. Django 


is Python’s most popular web framework, a set of tools 


designed for building interactive web applications. In 
this chapter, you'll learn how to use Django to build a project called Learning 
Log, an online journal system that lets you keep track of information you've 
learned about different topics. 

We'll write a specification for this project, and then define models for 
the data the app will work with. We'll use Django's admin system to enter 
some initial data, and then write views and templates so Django can build 
the site's pages. 

Django can respond to page requests and make it easier to read and 
write to a database, manage users, and much more. In Chapters 19 and 20, 
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you'll refine the Learning Log project, and then deploy it to a live server so 
you (and everyone else in the world) can use it. 


Setting Up a Project 


Chapter 18 


When starting work on something as significant as a web app, you first need to 
describe the project’s goals in a specification, or spec. Once you have a clear 
set of goals, you can start to identify manageable tasks to achieve those 
goals. 

In this section, we’ll write a spec for Learning Log and start working on 
the first phase of the project. This will involve setting up a virtual environ- 
ment and building out the initial aspects of a Django project. 


Writing a Spec 


A full spec details the project goals, describes the project’s functionality, 
and discusses its appearance and user interface. Like any good project or 
business plan, a spec should keep you focused and help keep your project 
on track. We won't write a full project spec here, but we'll lay out a few clear 
goals to keep the development process focused. Here's the spec we'll use: 


We'll write a web app called Learning Log that allows users to 
log the topics they're interested in and make journal entries as 
they learn about each topic. The Learning Log home page will 
describe the site and invite users to either register or log in. Once 
logged in, a user can create new topics, add new entries, and read 
and edit existing entries. 


When you're researching a new topic, maintaining a journal of what 
you've learned can help you keep track of new information and informa- 
tion you've already found. This is especially true when studying technical 
subjects. A good app, like the one we'll be creating, can help make this 
process more efficient. 


Creating a Virtual Environment 


To work with Django, we'll first set up a virtual environment. A virtual envi- 
ronment is a place on your system where you can install packages and isolate 
them from all other Python packages. Separating one project's libraries 
from other projects is beneficial and will be necessary when we deploy 
Learning Log to a server in Chapter 20. 

Create a new directory for your project called learning. log, switch to 
that directory in a terminal, and enter the following code to create a virtual 
environment: 


learning log$ python -m venv ll env 
learning log$ 


Here we're running the venv virtual environment module and using 
it to create an environment named // env (note that this name starts with 


two lowercase Ls, not two ones). If you use a command such as python3 
when running programs or installing packages, make sure to use that 
command here. 


Activating the Virtual Environment 


Now we need to activate the virtual environment, using the following 
command: 


learning log$ source 11 env/bin/activate 
(1l env)learning log$ 


This command runs the script activate in ll_env/bin/, When the environ- 
ment is active, you'll see the name of the environment in parentheses. This 
indicates that you can install new packages to the environment and use 
packages that have already been installed. Packages you install in //. env will 
not be available when the environment is inactive. 


If yow're using Windows, use the command 11_env\Scripts\activate (without the 
word source) to activate the virtual environment. If you're using PowerShell, you 


might need to capitalize Activate. 


To stop using a virtual environment, enter deactivate: 


(1l env)learning log$ deactivate 
learning log$ 


The environment will also become inactive when you close the terminal 
it’s running in. 


Installing Django 
With the virtual environment activated, enter the following to update pip 
and install Django: 


(1l env)learning log$ pip install --upgrade pip 

(1l env)learning log$ pip install django 

Collecting django 

--snip-- 

Installing collected packages: sqlparse, asgiref, django 
Successfully installed asgiref-3.5.2 django-4.1 sqlparse-0.4.2 
(1l env)learning log$ 


Because it downloads resources from a variety of sources, pip is upgraded 
fairly often. It's a good idea to upgrade pip whenever you make a new virtual 
environment. 

We're working in a virtual environment now, so the command to install 
Django is the same on all systems. There's no need to use longer com- 
mands, such as python -m pip install package name, or to include the --user 
flag. Keep in mind that Django will be available only when the // env envi- 
ronment is active. 
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Django releases a new version about every eight months, so you may see a newer ver- 
sion when you install Django. This project will most likely work as it’s written here, 
even on newer versions of Django. If you want to make sure to use the same version 
of Django you see here, use the command pip install django--4.1.*. This will 
install the latest release of Django 4.1. If you have any issues related to the version 
you're using, see the online resources for this book at https://ehmatthes.github.io/ 
pec. 3e. 


Creating a Project in Django 

Without leaving the active virtual environment (remember to look for //. eno 
in parentheses in the terminal prompt), enter the following commands to 
create a new project: 


© (11 env)learning log$ django-admin startproject ll project . 
6e (11 env)learning log$ ls 


1l env ll project manage.py 


© (11 env)learning log$ ls 1l project 


. init .py asgi.py settings.py urls.py wsgi.py 


The startproject command 6 tells Django to set up a new project called 
Il. project. The dot (.) at the end of the command creates the new project 
with a directory structure that will make it easy to deploy the app to a server 
when we're finished developing it. 


Don't forget this dot, or you might run into some configuration issues when you 
deploy the app. If you forget the dot, delete the files and folders that were created 
(except ll. env) and run the command again. 


Running the 1s command (dir on Windows) shows that Django has 
created a new directory called //. project. It also created a manage.py file, 
which is a short program that takes in commands and feeds them to the 
relevant part of Django. We'll use these commands to manage tasks, such as 
working with databases and running servers. 

The il. project directory contains four files ©; the most important are 
settings.py, urls.py, and wsgi.py. The settings.py file controls how Django inter- 
acts with your system and manages your project. We'll modify a few of these 
settings and add some settings of our own as the project evolves. The urls.py 
file tells Django which pages to build in response to browser requests. The 
wsgi.py file helps Django serve the files it creates. The filename is an acro- 
nym for *web server gateway interface." 


Creating the Database 


Django stores most of the information for a project in a database, so next 
we need to create a database that Django can work with. Enter the follow- 
ing command (still in an active environment): 


(1l env)learning log$ python manage.py migrate 


@ Operations to perform: 


Apply all migrations: admin, auth, contenttypes, sessions 
Running migrations: 
Applying contenttypes.0001 initial... OK 
Applying auth.0001 initial... OK 
--snip-- 
Applying sessions.0001 initial... OK 
6 (11 env)learning log$ 1s 
db.sqlite3 ll env ll project manage.py 


Anytime we modify a database, we say we're migrating the database. 
Issuing the migrate command for the first time tells Django to make sure the 
database matches the current state of the project. The first time we run this 
command in a new project using SOLite (more about SOLite in a moment), 
Django will create a new database for us. Here, Django reports that it will 
prepare the database to store information it needs to handle administrative 
and authentication tasks 6. 

Running the 1s command shows that Django created another file called 
db. sqlite? 9. SQLite is a database that runs off a single file; it's ideal for writ- 
ing simple apps because you won't have to pay much attention to managing 
the database. 


In an active virtual environment, use the command python to run manage.py com- 
mands, even if you use something different, like python3, to run other programs. In 
a virtual environment, the command python refers to the version of Python that was 
used to create the virtual environment. 


Viewing the Project 


Let’s make sure that Django has set up the project properly. Enter the 
runserver command to view the project in its current state: 


(1l env)learning log$ python manage.py runserver 
Watching for file changes with StatReloader 
Performing system checks... 


6 System check identified no issues (0 silenced). 
May 19, 2022 - 21:52:35 
@ Django version 4.1, using settings 'll project.settings' 
© Starting development server at http://127.0.0.1:8000/ 
Quit the server with CONTROL-C. 


Django should start a server called the development server, so you can 
view the project on your system to see how well it works. When you request 
a page by entering a URL in a browser, the Django server responds to that 
request by building the appropriate page and sending it to the browser. 

Django first checks to make sure the project is set up properly 8; it 
then reports the version of Django in use and the name of the settings file 
in use Ó. Finally, it reports the URL where the project is being served 8. 
The URL hitp://127.0.0.1:8000/ indicates that the project is listening for 
requests on port 8000 on your computer, which is called a localhost. The 
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term localhost refers to a server that only processes requests on your system; 
it doesn’t allow anyone else to see the pages you're developing. 

Open a web browser and enter the URL hitp://localhost:SO00/, or http://127 
.0.0.1:8000/ if the first one doesn’t work. You should see something like 
Figure 18-1: a page that Django creates to let you know everything is working 
properly so far. Keep the server running for now, but when you want to stop 
the server, press CTRL-C in the terminal where the runserver command was 


issued. 


django ease n Dja 
A 
pa 
The install worked successfully! Congratulations! 
Yo EBU ny 
figu RLs 
Django Documentation Tutorial: A Polling App Django Community 


Figure 18-1: Everything is working so far. 


If you receive the error message “That port is already in use,” tell Django to use a dif- 
ferent port by entering python manage.py runserver 8001 and then cycling through 
higher numbers until you find an open port. 


TRY IT YOURSELF 


18-1. New Projects: To get a better idea of what Django does, build a couple 
empty projects and look at what Django creates. Make a new folder with a 
simple name, like tik gram or insta. tok (outside of your learning. log directory], 


navigate to that folder in a terminal, and create a virtual environment. Install 
Django and run the command django-admin.py startproject tg project . 
(making sure to include the dot at the end of the command]. 

Look at the files and folders this command creates, and compare them to 
Learning Log. Do this a few times, until you're familiar with what Django creates 
when starting a new project. Then delete the project directories if you wish. 
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Starting an App 


A Django project is organized as a group of individual apps that work together 
to make the project work as a whole. For now, we’ll create one app to do 
most of our project's work. We'll add another app in Chapter 19 to manage 
user accounts. 

You should leave the development server running in the terminal win- 
dow you opened earlier. Open a new terminal window (or tab) and navigate 
to the directory that contains manage.py. Activate the virtual environment, 
and then run the startapp command: 


learning log$ source 1l env/bin/activate 
(1l env)learning log$ python manage.py startapp learning logs 


© (11 env)learning log$ 1s 


db.sqlite3 learning logs ll env ll project manage.py 


@ (11 env)learning log$ ls learning logs/ 


models.py 


e 
e 


. init .py admin.py apps.py migrations models.py tests.py views.py 


The command startapp appname tells Django to create the infrastructure 
needed to build an app. When you look in the project directory now, you'll 
see a new folder called learning. logs €. Use the 1s command to see what 
Django has created @. The most important files are models.py, admin.py, and 
views.py. We'll use models.py to define the data we want to manage in our 
app. We'll look at admin.py and views.py a little later. 


Defining Models 


Let's think about our data for a moment. Each user will need to create a 

number of topics in their learning log. Each entry they make will be tied to a 

topic, and these entries will be displayed as text. We'll also need to store the 

timestamp of each entry so we can show users when they made each one. 
Open the file models.py and look at its existing content: 


from django.db import models 


# Create your models here. 


A module called models is being imported, and we're being invited to 
create models of our own. A model tells Django how to work with the data 
that will be stored in the app. A model is a class; it has attributes and meth- 
ods, just like every class we've discussed. Here's the model for the topics 
users will store: 


class Topic(models.Model): 
"""A topic the user is learning about.""" 
text - models.CharField(max length-200) 
date added - models.DateTimeField(auto now add-True) 
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def str (self): 
"""Return a string representation of the model.""" 
return self.text 


We've created a class called Topic, which inherits from Model—a parent 
class included in Django that defines a model’s basic functionality. We add 
two attributes to the Topic class: text and date added. 

The text attribute is a CharField, a piece of data that's made up of char- 
acters or text €. You use CharField when you want to store a small amount of 
text, such as a name, a title, or a city. When we define a CharField attribute, 
we have to tell Django how much space it should reserve in the database. 
Here we give it a max length of 200 characters, which should be enough to 
hold most topic names. 

The date added attribute is a DateTimeField, a piece of data that will 
record a date and time 6. We pass the argument auto now add-True, which 
tells Django to automatically set this attribute to the current date and time 
whenever the user creates a new topic. 

It's a good idea to tell Django how you want it to represent an instance 
of a model. If a model hasa str () method, Django calls that method 
whenever it needs to generate output referring to an instance of that model. 
Here we've writtena. str. () method that returns the value assigned to the 
text attribute ©. 

To see the different kinds of fields you can use in a model, see the 
“Model Field Reference" page at https://docs.djangoproject.com/en/4.1/ref/ 
models/fields. You won't need all the information right now, but it will be 
extremely useful when you're developing your own Django projects. 


Activating Models 


To use our models, we have to tell Django to include our app in the overall 
project. Open settings.py (in the ll_project directory); you'll see a section that 
tells Django which apps are installed in the project: 


--snip-- 

INSTALLED APPS = [ 
'django.contrib.admin', 
'django.contrib.auth', 
'django.contrib.contenttypes', 
'django.contrib.sessions', 
'django.contrib.messages', 
'django.contrib.staticfiles', 


] 


--snip-- 


Add our app to this list by modifying INSTALLED APPS so it looks like this: 


# My apps. 
'learning logs', 


# Default django apps. 


Grouping apps together in a project helps keep track of them as the 
project grows to include more apps. Here we start a section called My apps, 
which includes only ‘learning logs' for now. It's important to place your own 
apps before the default apps, in case you need to override any behavior of 
the default apps with your own custom behavior. 

Next, we need to tell Django to modify the database so it can store 
information related to the model Topic. From the terminal, run the follow- 
ing command: 


(1l env)learning log$ python manage.py makemigrations learning logs 
Migrations for 'learning logs': 
learning logs/migrations/0001 initial.py 
- Create model Topic 
(1l env)learning log$ 


The command makemigrations tells Django to figure out how to modify 
the database so it can store the data associated with any new models we've 
defined. The output here shows that Django has created a migration file 
called 0001 initial.py. This migration will create a table for the model Topic 
in the database. 

Now we'll apply this migration and have Django modify the database 
for us: 


(1l env)learning log$ python manage.py migrate 
Operations to perform: 

Apply all migrations: admin, auth, contenttypes, learning logs, sessions 
Running migrations: 

Applying learning logs.0001 initial... OK 


Most of the output from this command is identical to the output from 
the first time we issued the migrate command. We need to check the last line 
in this output, where Django confirms that the migration for learning logs 
worked OK. 

Whenever we want to modify the data that Learning Log manages, we'll 
follow these three steps: modify models.py, call makemigrations on learning logs, 
and tell Django to migrate the project. 


The Django Admin Site 


Django makes it easy to work with your models through its admin site. Django's 
admin site is only meant to be used by the site's administrators; it’s not meant for 
regular users. In this section, we'll set up the admin site and use it to add some 
topics through the Topic model. 
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Setting Up a Superuser 


Django allows you to create a superuser, a user who has all privileges avail- 
able on the site. A user’s privileges control the actions they can take. The 
most restrictive privilege settings allow a user to only read public informa- 
tion on the site. Registered users typically have the privilege of reading 
their own private data and some selected information available only to 
members. To effectively administer a project, the site owner usually needs 
access to all information stored on the site. A good administrator is careful 
with their users’ sensitive information, because users put a lot of trust into 
the apps they access. 

To create a superuser in Django, enter the following command and 
respond to the prompts: 


(1l env)learning log$ python manage.py createsuperuser 


@ Username (leave blank to use 'eric'): ll admin 
© Email address: 
© Password: 


admin.py 
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Password (again): 
Superuser created successfully. 
(1l env)learning log$ 


When you issue the command createsuperuser, Django prompts you to 
enter a username for the superuser 6. Here I’m using 11 admin, but you can 
enter any username you want. You can enter an email address or just leave 
this field blank @. You'll need to enter your password twice 6. 


Some sensitive information can be hidden from a site's administrators. For example, 
Django doesn't store the password you enter; instead, it stores a string derived from 
the password, called a hash. Each time you enter your password, Django hashes your 
entry and compares it to the stored hash. If the two hashes match, you’re authenti- 
cated. By requiring hashes to match, Django ensures that if an attacker gains access to 
a site's database, they'll be able to read the stored hashes but not the passwords. When 
a site is set up properly, it’s almost impossible to get the original passwords from the 
hashes. 


Registering a Model with the Admin Site 


Django includes some models in the admin site automatically, such as User 
and Group, but the models we create need to be added manually. 

When we started the learning logs app, Django created an admin.py file 
in the same directory as models.py. Open the admin.py file: 


from django.contrib import admin 


# Register your models here. 


To register Topic with the admin site, enter the following: 
from django.contrib import admin 
from .models import Topic 
admin.site.register(Topic) 


This code first imports the model we want to register, Topic. The dot 
in front of models tells Django to look for models.py in the same directory as 
admin.py. The code admin.site.register() tells Django to manage our model 
through the admin site. 

Now use the superuser account to access the admin site. Go to http:// 
localhost:8000/admin/ and enter the username and password for the super- 
user you just created. You should see a screen similar to the one shown in 
Figure 18-2. This page allows you to add new users and groups, and change 
existing ones. You can also work with data related to the Topic model that 
we just defined. 


eoe D < (& locathost:8000/admin) te c) © Ô + 


o | 


Site administration | Django site admin 


Dja ngo a dministration WELCOME, LL. ADMIN. VIEW SITE / CHANGE PASSWORD / LOG OUT 


Site administration 


AUTHENTICATION AND AUTHORIZATION s 
Recent actions 
Groups +Add 2 Change 
Users +Add — 4 Change My actions 
None available 
LEARNING_LOGS 


Topics +Add — 4 Change 


Figure 18-2: The admin site with Topic included 


If you see a message in your browser that the web page is not available, make sure you 
still have the Django server running in a terminal window. If you don't, activate a 
virtual environment and reissue the command python manage.py runserver. If yow're 
having trouble viewing your project at amy point in the development process, closing any 

open terminals and reissuing the runserver command is a good first troubleshooting step. 


Adding Topics 
Now that Topic has been registered with the admin site, let's add our first 
topic. Click Topics to go to the Topics page, which is mostly empty, because 
we have no topics to manage yet. Click Add Topic, and a form for adding a 
new topic appears. Enter Chess in the first box and click Save. You'll be sent 
back to the Topics admin page, and you'll see the topic you just created. 
Let's create a second topic so we'll have more data to work with. 
Click Add Topic again, and enter Rock Climbing. Click Save, and you'll be 
sent back to the main Topics page again. Now you'll see Chess and Rock 
Climbing listed. 
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Defining the Entry Model 


For a user to record what they’ve been learning about chess and rock climb- 
ing, we need to define a model for the kinds of entries users can make in 
their learning logs. Each entry needs to be associated with a particular 
topic. This relationship is called a many-to-one relationship, meaning many 
entries can be associated with one topic. 

Here’s the code for the Entry model. Place it in your models.py file: 


© class Entry(models.Model): 


Something specific learned about a topic. 
topic - models.ForeignKey(Topic, on delete-models.CASCADE) 
text - models.TextField() 

date added - models.DateTimeField(auto now add-True) 


class Meta: 
verbose name plural = 'entries' 


def str (self): 
"""Return a simple string representing the entry. 
return f"{self.text[:50]}..." 


The Entry class inherits from Django's base Model class, just as Topic 
did €. The first attribute, topic, is a ForeignKey instance @. A foreign key is a 
database term; it's a reference to another record in the database. This is the 
code that connects each entry to a specific topic. Each topic is assigned a 
key, or ID, when it's created. When Django needs to establish a connection 
between two pieces of data, it uses the keys associated with each piece of 
information. We'll use these connections shortly to retrieve all the entries 
associated with a certain topic. The on delete-models.CASCADE argument tells 
Django that when a topic is deleted, all the entries associated with that topic 
should be deleted as well. This is known as a cascading delete. 

Next is an attribute called text, which is an instance of TextField 9. 
This kind of field doesn't need a size limit, because we don't want to limit 
the size of individual entries. The date added attribute allows us to present 
entries in the order they were created, and to place a timestamp next to 
each entry. 

The Meta class is nested inside the Entry class O. The Meta class holds 
extra information for managing a model; here, it lets us set a special attri- 
bute telling Django to use Entries when it needs to refer to more than one 
entry. Without this, Django would refer to multiple entries as Entrys. 

The str () method tells Django which information to show when 
it refers to individual entries. Because an entry can be a long body of text, 

. str ()returns just the first 50 characters of text 69. We also add an ellipsis 
to clarify that we're not always displaying the entire entry. 


Migrating the Entry Model 


Because we’ve added a new model, we need to migrate the database again. 
This process will become quite familiar: you modify models.py, run the com- 
mand python manage.py makemigrations app_name, and then run the command 
python manage.py migrate. 

Migrate the database and check the output by entering the following 
commands: 


(1l env)learning log$ python manage.py makemigrations learning logs 
Migrations for 'learning logs': 
€ learning logs/migrations/0002 entry.py 
- Create model Entry 
(1l env)learning log$ python manage.py migrate 
Operations to perform: 
--snip-- 
€ Applying learning logs.0002 entry... OK 


A new migration called 0002 entry.py is generated, which tells Django 
how to modify the database to store information related to the model 
Entry 60. When we issue the migrate command, we see that Django applied 
this migration and everything worked properly 6. 


Registering Entry with the Admin Site 


We also need to register the Entry model. Here's what admin.py should look 
like now: 


admin.py 


from .models import Topic, Entry 


admin.site. register(Entry) 


Go back to hittp://localhost/admin/, and you should see Entries listed 
under Learning. Logs. Click the Add link for Entries, or click Entries and 
then choose Add entry. You should see a drop-down list to select the topic 
you're creating an entry for and a text box for adding an entry. Select Chess 
from the drop-down list, and add an entry. Here's the first entry I made: 


The opening is the first part of the game, roughly the first ten 
moves or so. In the opening, it's a good idea to do three things— 
bring out your bishops and knights, try to control the center of 
the board, and castle your king. 


Of course, these are just guidelines. It will be important to learn 


when to follow these guidelines and when to disregard these 
suggestions. 


When you click Save, you'll be brought back to the main admin page 
for entries. Here, you'll see the benefit of using text[:50] as the string rep- 
resentation for each entry; it's much easier to work with multiple entries in 
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the admin interface if you see only the first part of an entry, rather than the 
entire text of each entry. 

Make a second entry for Chess and one entry for Rock Climbing so we 
have some initial data. Here’s a second entry for Chess: 


In the opening phase of the game, it’s important to bring out 
your bishops and knights. These pieces are powerful and maneu- 
verable enough to play a significant role in the beginning moves 
of a game. 


And here’s a first entry for Rock Climbing: 


One of the most important concepts in climbing is to keep your 
weight on your feet as much as possible. There’s a myth that 
climbers can hang all day on their arms. In reality, good climb- 
ers have practiced specific ways of keeping their weight over their 
feet whenever possible. 


These three entries will give us something to work with as we continue 
to develop Learning Log. 


The Django Shell 


Now that we’ve entered some data, we can examine it programmatically 
through an interactive terminal session. This interactive environment is 
called the Django shell, and it’s a great environment for testing and trouble- 
shooting your project. Here’s an example of an interactive shell session: 


(11_env)learning log$ python manage.py shell 
@ >>> from learning logs.models import Topic 
»»» Topic.objects.all() 
<QuerySet [<Topic: Chess», «Topic: Rock Climbing>]> 


The command python manage.py shell, run in an active virtual environ- 
ment, launches a Python interpreter that you can use to explore the data 
stored in your project's database. Here, we import the model Topic from 
the learning logs.models module 6.. We then use the method Topic.objects 
.all() to get all instances of the model Topic; the list that's returned is 
called a queryset. 

We can loop over a queryset just as we'd loop over a list. Here's how you 
can see the ID that's been assigned to each topic object: 


»»» topics - Topic.objects.all() 
»»» for topic in topics: 
print(topic.id, topic) 


1 Chess 
2 Rock Climbing 


We assign the queryset to topics and then print each topic's id attribute 
and the string representation of each topic. We can see that Chess has an ID 
of 1 and Rock Climbing has an ID of 2. 
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If you know the ID of a particular object, you can use the method Topic 
-objects.get() to retrieve that object and examine any attribute the object 
has. Let's look at the text and date added values for Chess: 


»»» t - Topic.objects.get(id-1) 

>>> t.text 

"Chess' 

>>> t.date_added 

datetime.datetime(2022, 5, 20, 3, 33, 36, 928759, 
tzinfo-datetime.timezone.utc) 


We can also look at the entries related to a certain topic. Earlier, we 
defined the topic attribute for the Entry model. This was a ForeignKey, a con- 
nection between each entry and a topic. Django can use this connection to 
get every entry related to a certain topic, like this: 


»»» t.entry set.all() 

<QuerySet [<Entry: The opening is the first part of the game, roughly...>, 
«Entry: 

In the opening phase of the game, it's important t...>]> 


To get data through a foreign key relationship, you use the lowercase 
name of the related model followed by an underscore and the word set 6. 
For example, say you have the models Pizza and Topping, and Topping is 
related to Pizza through a foreign key. If your object is called my pizza, repre- 
senting a single pizza, you can get all of the pizza's toppings using the code 
my pizza.topping set.all(). 

We'll use this syntax when we begin to code the pages users can 
request. The shell is really useful for making sure your code retrieves the 
data you want it to. If your code works as you expect it to in the shell, it 
should also work properly in the files within your project. If your code gen- 
erates errors or doesn't retrieve the data you expect it to, it’s much easier 
to troubleshoot your code in the simple shell environment than within the 
files that generate web pages. We won't refer to the shell much, but you 
should continue using it to practice working with Django's syntax for access- 
ing the data stored in the project. 

Each time you modify your models, you'll need to restart the shell to 
see the effects of those changes. To exit a shell session, press CTRL-D; on 
Windows, press CTRL-Z and then press ENTER. 


TRY IT YOURSELF 


18-2. Short Entries: The — str () method in the Entry model currently appends 


an ellipsis to every instance of Entry when Django shows it in the admin site 
or the shell. Add an if statement to the str () method that adds an ellipsis 


[continued] 


Getting Started with Django 387 


388 


only if the entry is longer than 50 characters. Use the admin site to add an 
entry that’s fewer than 50 characters in length, and check that it doesn’t have 
an ellipsis when viewed. 


18-3. The Django API: When you write code to access the data in your project, 
you're writing a query. Skim through the documentation for querying your data 
at https://docs.djangoproject.com/en/4. l/topics/db/queries. Much of what you 
see will look new to you, but it will be quite useful as you start to work on your 
own projects. 

18-4. Pizzeria: Start a new project called pizzeria project with an app called 
pizzas. Define a model Pizza with a field called name, which will hold name 
values, such as Hawaiian and Meat Lovers. Define a model called Topping with 


fields called pizza and name. The pizza field should be a foreign key to Pizza, 


and name should be able to hold values such as pineapple, Canadian bacon, and 
sausage. 

Register both models with the admin site, and use the site to enter some 
pizza names and toppings. Use the shell to explore the data you entered. 


Making Pages: The Learning Log Home Page 
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Making web pages with Django consists of three stages: defining URLs, 
writing views, and writing templates. You can do these in any order, but in 
this project we'll always start by defining the URL pattern. A URL pattern 
describes the way the URL is laid out. It also tells Django what to look for 
when matching a browser request with a site URL, so it knows which page 
to return. 

Each URL then maps to a particular view. The viewfunction retrieves 
and processes the data needed for that page. The view function often ren- 
ders the page using a template, which contains the overall structure of the 
page. To see how this works, let's make the home page for Learning Log. 
We'll define the URL for the home page, write its view function, and create 
a simple template. 

Because we just want to ensure that Learning Log works as it's supposed 
to, we'll make a simple page for now. A functioning web app is fun to style 
when it's complete; an app that looks good but doesn't work well is point- 
less. For now, the home page will display only a title and a brief description. 


Mapping a URL 

Users request pages by entering URLs into a browser and clicking links, so 
we'll need to decide what URLs are needed. The home page URL is first: 
it's the base URL people use to access the project. At the moment, the base 
URL, Attp://localhost:8000/, returns the default Django site that lets us know 
the project was set up correctly. We'll change this by mapping the base URL 
to Learning Log's home page. 


In the main Jl_project folder, open the file urls.py. You should see the fol- 
lowing code: 


l| project/ € from django.contrib import admin 
urlpy from django.urls import path 


@ urlpatterns = [ 
e path('admin/', admin.site.urls), 
] 


The first two lines import the admin module and a function to build URL 
paths 0. The body of the file defines the urlpatterns variable @. In this urls.py 
file, which defines URLs for the project as a whole, the urlpatterns variable 
includes sets of URLs from the apps in the project. The list includes the 
module admin.site.urls, which defines all the URLs that can be requested 
from the admin site 9. 

We need to include the URLs for learning logs, so add the following: 


from django.urls import path, include 


path('', include('learning logs.urls')), 


We've imported the include() function, and we've also added a line to 
include the module learning logs.urls. 

The default urls.py is in the //. project folder; now we need to make a sec- 
ond urls.py file in the learning. logs folder. Create a new Python file, save it as 
urls.py in learning. logs, and enter this code into it: 


learning logs/ ® """Defines URL patterns for learning logs. 


urls.py 
@ from django.urls import path 


© from . import views 


O app name - 'learning logs' 
© urlpatterns = [ 
# Home page 
[6) path('', views.index, name-'index'), 


] 


To make it clear which urls.py we're working in, we add a docstring at 
the beginning of the file €. We then import the path function, which is 
needed when mapping URLs to views 9. We also import the views mod- 
ule 6; the dot tells Python to import the views.py module from the same 
directory as the current urls.py module. The variable app name helps Django 
distinguish this urls.py file from files of the same name in other apps within 
the project O. The variable urlpatterns in this module is a list of individual 
pages that can be requested from the learning logs app 9. 
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The actual URL pattern is a call to the path() function, which takes 
three arguments @. The first argument is a string that helps Django route 
the current request properly. Django receives the requested URL and tries 
to route the request to a view. It does this by searching all the URL patterns 
we've defined to find one that matches the current request. Django ignores 
the base URL for the project (Attp://localhost:8000/), so the empty string 
(^ ') matches the base URL. Any other URL won't match this pattern, and 
Django will return an error page if the URL requested doesn't match any 
existing URL patterns. 

The second argument in path() O specifies which function to call 
in views.py. When a requested URL matches the pattern we're defining, 
Django calls the index() function from views.py. (We'll write this view func- 
tion in the next section.) The third argument provides the name index for 
this URL pattern so we can refer to it more easily in other files throughout 
the project. Whenever we want to provide a link to the home page, we'll use 
this name instead of writing out a URL. 


Writing a View 

A view function takes in information from a request, prepares the data 

needed to generate a page, and then sends the data back to the browser. It 

often does this by using a template that defines what the page will look like. 
The file views.py in learning. logs was generated automatically when we ran 

the command python manage.py startapp. Here's what's in views.py right now: 


from django.shortcuts import render 


# Create your views here. 


Currently, this file just imports the render() function, which renders the 
response based on the data provided by views. Open views.py and add the 
following code for the home page: 


def index(request): 
"""The home page for Learning Log. 
return render(request, 'learning logs/index.html') 


When a URL request matches the pattern we just defined, Django looks 
for a function called index() in the views.py file. Django then passes the 
request object to this view function. In this case, we don't need to process any 
data for the page, so the only code in the function is a call to render(). The 
render() function here passes two arguments: the original request object and 
a template it can use to build the page. Let's write this template. 


Writing a Template 


The template defines what the page should look like, and Django fills in 
the relevant data each time the page is requested. A template allows you to 


index.html 


access any data provided by the view. Because our view for the home page 
provides no data, this template is fairly simple. 

Inside the learning_logs folder, make a new folder called templates. 
Inside the templates folder, make another folder called learning_logs. This 
might seem a little redundant (we have a folder named learning_logs inside a 
folder named templates inside a folder named learning_logs), but it sets up a 
structure that Django can interpret unambiguously, even in the context of a 
large project containing many individual apps. Inside the inner learning_logs 
folder, make a new file called index.html. The path to the file will be //. project/ 
learning. logs/templates/learning. logs/index.html. Enter the following code into 
that file: 


<p>Learning Log</p> 


<p>Learning Log helps you keep track of your learning, for any topic you're 
interested in.</p> 


This is a very simple file. If you’re not familiar with HTML, the <p></p> 
tags signify paragraphs. The <p> tag opens a paragraph, and the </p> tag 
closes a paragraph. We have two paragraphs: the first acts as a title, and the 
second describes what users can do with Learning Log. 

Now when you request the project's base URL, Attp:;//localhost:8000/, you 
should see the page we just built instead of the default Django page. Django 
will take the requested URL, and that URL will match the pattern ''; then 
Django will call the function views.index(), which will render the page using 
the template contained in index.html. Figure 18-3 shows the resulting page. 


eee D < @ localhost:8000 ; © + B 
localhost:8000 
Learning Log 


Learning Log helps you keep track of your learning, for any topic you're interested in. 


Figure 18-3: The home page for Learning Log 


Although it might seem like a complicated process for creating one 
page, this separation between URLs, views, and templates works quite well. It 
allows you to think about each aspect of a project separately. In larger proj- 
ects, it allows individuals working on the project to focus on the areas in 
which they’re strongest. For example, a database specialist can focus on the 
models, a programmer can focus on the view code, and a frontend special- 
ist can focus on the templates. 
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You might see the following error message: 

ModuleNotFoundError: No module named ‘learning logs.urls' 

If you do, stop the development server by pressing CTRL-C in the terminal win- 
dow where you issued the runserver command. Then reissue the command python 
manage.py runserver. You should be able to see the home page. Anytime you run into 
an error like this, try stopping and restarting the server. 


TRY IT YOURSELF 


18-5. Meal Planner: Consider an app that helps people plan their meals 
throughout the week. Make a new folder called meal_planner, and start 
a new Django project inside this folder. Then make a new app called meal 


_plans. Make a simple home page for this project. 


18-6. Pizzeria Home Page: Add a home page to the Pizzeria project you 
started in Exercise 18-4 (page 388). 


Building Additional Pages 


Now that we’ve established a routine for building a page, we can start to 
build out the Learning Log project. We’ll build two pages that display data: 
a page that lists all topics and a page that shows all the entries for a particu- 
lar topic. For each page, we’ll specify a URL pattern, write a view function, 
and write a template. But before we do this, we’ll create a base template 
that all templates in the project can inherit from. 


Template Inheritance 


When building a website, some elements will need to be repeated on each 
page. Rather than writing these elements directly into each page, you can 
write a base template containing the repeated elements and then have each 
page inherit from the base. This approach lets you focus on developing the 
unique aspects of each page, and makes it much easier to change the over- 
all look and feel of the project. 


The Parent Template 


We'll create a template called base.himlin the same directory as index.html. 
This file will contain elements common to all pages; every other template 
will inherit from base.himl. The only element we want to repeat on each 
page right now is the title at the top. Because we'll include this template 
on every page, let's make the title a link to the home page: 


base.html <p> 
© <a href="{% url ‘learning logs:index' %}">Learning Log</a> 
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</p> 


@ {% block content %}{% endblock content %} 


The first part of this file creates a paragraph containing the name of 
the project, which also acts as a home page link. To generate a link, we use 
a template tag, which is indicated by braces and percent signs ({% %}). A tem- 
plate tag generates information to be displayed on a page. The template 
tag {% url ‘learning logs:index' %} shown here generates a URL matching 
the URL pattern defined in learning_logs/urls.py with the name 'index' @. In 
this example, learning logs is the namespace and index is a uniquely named 
URL pattern in that namespace. The namespace comes from the value we 
assigned to app_name in the learning_logs/urls.py file. 

In a simple HTML page, a link is surrounded by the anchor tag «a»: 


«a href-"link url"»link text</a> 


Having the template tag generate the URL for us makes it much easier 
to keep our links up to date. We only need to change the URL pattern in 
urls.py, and Django will automatically insert the updated URL the next time 
the page is requested. Every page in our project will inherit from base.html, 
so from now on, every page will have a link back to the home page. 

On the last line, we insert a pair of block tags @. This block, named content, 
is a placeholder; the child template will define the kind of information that 
goes in the content block. 

A child template doesn't have to define every block from its parent, so 
you can reserve space in parent templates for as many blocks as you like; the 
child template uses only as many as it needs. 


In Python code, we almost always use four spaces when we indent. Template files tend 
to have more levels of nesting than Python files, so it’s common to use only two spaces 
for each indentation level. 


The Child Template 


Now we need to rewrite index.html to inherit from base.html. Add the follow- 
ing code to index.html: 


index.html @ {% extends 'learning logs/base.html' %} 


@ {% block content X) 


e {% endblock content %} 


If you compare this to the original index.html, you can see that we've 
replaced the Learning Log title with the code for inheriting from a parent 
template 6. A child template must have an {% extends %} tag on the first 
line to tell Django which parent template to inherit from. The file base.himl 
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is part of learning logs, so we include learning_logs in the path to the parent 
template. This line pulls in everything contained in the base.himl template 
and allows index.himl to define what goes in the space reserved by the content 
block. 

We define the content block by inserting a {% block %} tag with the 
name content @. Everything that we aren’t inheriting from the parent tem- 
plate goes inside the content block. Here, that’s the paragraph describing 
the Learning Log project. We indicate that we’re finished defining the con- 
tent by using an {% endblock content %} tag ©. The {% endblock %} tag doesn't 
require a name, but if a template grows to contain multiple blocks, it can be 
helpful to know exactly which block is ending. 

You can start to see the benefit of template inheritance: in a child 
template, we only need to include content that’s unique to that page. This 
not only simplifies each template, but also makes it much easier to modify 
the site. To modify an element common to many pages, you only need to 
modify the parent template. Your changes are then carried over to every 
page that inherits from that template. In a project that includes tens or 
hundreds of pages, this structure can make it much easier and faster to 
improve your site. 

In a large project, it’s common to have one parent template called base 
-himl for the entire site and parent templates for each major section of the 
site. All the section templates inherit from base.himl, and each page in 
the site inherits from a section template. This way you can easily modify the 
look and feel of the site as a whole, any section in the site, or any individual 
page. This configuration provides a very efficient way to work, and encour- 
ages you to steadily update your project over time. 


The Topics Page 


Now that we have an efficient approach to building pages, we can focus on 
our next two pages: the general topics page and the page to display entries 
for a single topic. The topics page will show all topics that users have cre- 
ated, and it’s the first page that will involve working with data. 


The Topics URL Pattern 


First, we define the URL for the topics page. It’s common to choose a sim- 
ple URL fragment that reflects the kind of information presented on the 

page. We'll use the word topics, so the URL http://localhost:8000/topics/ will 

return this page. Here's how we modify learning. logs/urls.py: 


l 


# Page that shows all topics. 
path('topics/', views.topics, name='topics'), 


views. py 


topics. html 


LU 


e 


e 
o 
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The new URL pattern is the word topics, followed by a forward slash. 
When Django examines a requested URL, this pattern will match any URL 
that has the base URL followed by topics. You can include or omit a forward 
slash at the end, but there can't be anything else after the word topics, or the 
pattern won’t match. Any request with a URL that matches this pattern will 
then be passed to the function topics() in views.py. 


The Topics View 


The topics() function needs to retrieve some data from the database and 
send it to the template. Add the following to views.py: 


from .models import Topic 


dey 


def topics(request): 
"""Show all topics. 
topics = Topic.objects.order by('date added') 
context - ('topics': topics] 
return render(request, 'learning logs/topics.html', context) 


We first import the model associated with the data we need 6. The 
topics() function needs one parameter: the request object Django received 
from the server 0. We query the database by asking for the Topic objects, 
sorted by the date added attribute 6. We assign the resulting queryset to 
topics. 

We then define a context that we'll send to the template O. A contextis 
a dictionary in which the keys are names we'll use in the template to access 
the data we want, and the values are the data we need to send to the tem- 
plate. In this case, there's one key-value pair, which contains the set of topics 
we'll display on the page. When building a page that uses data, we call 
render() with the request object, the template we want to use, and the 
context dictionary 9. 


The Topics Template 


The template for the topics page receives the context dictionary, so the tem- 
plate can use the data that topics() provides. Make a file called topics.htmlin 
the same directory as index.html. Here's how we can display the topics in the 
template: 


{% extends ‘learning logs/base.html' X) 
{% block content %} 


<p>Topics</p> 
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<ul> 
{% for topic in topics %} 
<li>{{ topic.text }}</li> 
(^ empty %} 
«li»No topics have been added yet.«/li» 
{% endfor 4j 
</ul> 


oo C0086 


{% endblock content %} 


We use the {% extends %} tag to inherit from base.himl, just as we did on 
the home page, and then we open a content block. The body of this page 
contains a bulleted list of the topics that have been entered. In standard 
HTML, a bulleted list is called an unordered list and is indicated by the tags 
<ul></ul>. The opening tag <ul> begins the bulleted list of topics @. 

Next we use a template tag that’s equivalent to a for loop, which loops 
through the list topics from the context dictionary @. The code used in tem- 
plates differs from Python in some important ways. Python uses indentation 
to indicate which lines of a for statement are part of a loop. In a template, 
every for loop needs an explicit (4 endfor %} tag indicating where the end of 
the loop occurs. So in a template, you'll see loops written like this: 


{% for item in list %} 
do something with each item 
{% endfor %} 


Inside the loop, we want to turn each topic into an item in the bulleted 
list. To print a variable in a template, wrap the variable name in double 
braces. The braces won’t appear on the page; they just indicate to Django 
that we're using a template variable. So the code (( topic.text )) © will 
be replaced by the value of the current topic's text attribute on each pass 
through the loop. The HTML tag <1i></1i> indicates a list item. Anything 
between these tags, inside a pair of <ul></ul> tags, will appear as a bulleted 
item in the list. 

We also use the (4 empty 4) template tag @, which tells Django what to 
do if there are no items in the list. In this case, we print a message inform- 
ing the user that no topics have been added yet. The last two lines close out 
the for loop ® and then close out the bulleted list O. 

Now we need to modify the base template to include a link to the topics 
page. Add the following code to base.himl: 


base.html 
© <a href="{% url ‘learning logs:index' %}">Learning Log</a> - 
@ <a href="{% url ‘learning logs:topics' %}">Topics</a> 


We add a dash after the link to the home page @, and then add a link 
to the topics page using the {% url %} template tag again @. This line tells 
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Django to generate a link matching the URL pattern with the name 'topics' 
in learning. logs/urls.py. 

Now when you refresh the home page in your browser, you'll see a 
Topics link. When you click the link, you'll see a page that looks similar to 
Figure 18-4. 


pee D < ® localhost:8000/topics/ : © Ô + B 
localhost:8000/topics/ 
Learning Log - Topics 
Topics 


* Chess 
* Rock Climbing 


Figure 18-4: The topics page 


Individual Topic Pages 


Next, we need to create a page that can focus on a single topic, showing the 
topic name and all the entries for that topic. We'll define a new URL pat- 
tern, write a view, and create a template. We'll also modify the topics page 
so each item in the bulleted list links to its corresponding topic page. 


The Topic URL Pattern 


The URL pattern for the topic page isa little different from the prior 
URL patterns because it will use the topic's id attribute to indicate which 
topic was requested. For example, if the user wants to see the detail page 
for the Chess topic (where the id is 1), the URL will be http://locathost: 
8000/topics/1/. Here's a pattern to match this URL, which you should place 
in learning. logs/urls.py: 


erns 


# Detail page for a single topic. 
path('topics/«int:topic id»/', views.topic, name-'topic'), 


Let's examine the string 'topics/«int:topic id»/' in this URL pattern. 
The first part of the string tells Django to look for URLs that have the word 
topics after the base URL. The second part of the string, /<int:topic_id>/, 
matches an integer between two forward slashes and assigns the integer 
value to an argument called topic id. 
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When Django finds a URL that matches this pattern, it calls the view 
function topic() with the value assigned to topic id as an argument. We'll 
use the value of topic id to get the correct topic inside the function. 


The Topic View 


The topic() function needs to get the topic and all associated entries from 
the database, much like what we did earlier in the Django shell: 


--snip-- 

def topic(request, topic id): 
"""Show a single topic and all its entries. 
topic - Topic.objects.get(id-topic id) 
entries - topic.entry set.order by('-date added') 
context - ('topic': topic, 'entries': entries] 
return render(request, 'learning logs/topic.html', context) 


This is the first view function that requires a parameter other than the 
request object. The function accepts the value captured by the expression 
/<int:topic_id>/ and assigns it to topic id €. Then we use get() to retrieve 
the topic, just as we did in the Django shell 60. Next, we get all of the entries 
associated with this topic and order them according to date added ©. The 
minus sign in front of date added sorts the results in reverse order, which will 
display the most recent entries first. We store the topic and entries in the 
context dictionary @ and call render() with the request object, the topic.html 
template, and the context dictionary 9. 


The code phrases at @ and ® are called queries, because they query the database for 
specific information. When you're writing queries like these in your own projects, it’s 
helpful to try them out in the Django shell first. Yowll get much quicker feedback in 
the shell than you would by writing a view and template and then checking the results 
in a browser. 


The Topic Template 


The template needs to display the name of the topic and the entries. We also 
need to inform the user if no entries have been made yet for this topic. 


{% extends 'learning logs/base.html' %} 
{% block content 4) 
<p>Topic: {{ topic.text }}</p> 


<p>Entries:</p> 
<ul> 
{% for entry in entries %} 
<li> 
<p>{{ entry.date added|date:'M d, Y H:i' }}</p> 
<p>{{ entry.text|linebreaks }}</p> 
</li> 
{% empty %} 


topics. html 


<li>There are no entries for this topic yet.</li> 
{% endfor %} 
</ul> 


{% endblock content %} 


We extend base.himl, as we'll do for all pages in the project. Next, we 
show the text attribute of the topic that's been requested 6. The vari- 
able topic is available because it's included in the context dictionary. We 
then start a bulleted list € to show each of the entries and loop through 
them 6, as we did with the topics earlier. 

Each bullet lists two pieces of information: the timestamp and the full 
text of each entry. For the timestamp 9, we display the value of the attribute 
date added. In Django templates, a vertical line (|) represents a template filter— 
a function that modifies the value in a template variable during the rendering 
process. The filter date:'M d, Y H:i' displays timestamps in the format January 1, 
2022 23:00. The next line displays the value of the current entry's text attri- 
bute. The filter linebreaks 6 ensures that long text entries include line breaks 
in a format understood by browsers, rather than showing a block of uninter- 
rupted text. We again use the {% empty %} template tag © to print a message 
informing the user that no entries have been made. 


Links from the Topics Page 


Before we look at the topic page in a browser, we need to modify the topics 
template so each topic links to the appropriate page. Here's the change you 
need to make to topics. html: 


di» 
<a href="{% url 'learning logs:topic' topic.id %}"> 
{{ topic.text }}</a></li> 
</li> 


We use the URL template tag to generate the proper link, based on 
the URL pattern in learning_logs with the name 'topic'. This URL pattern 
requires a topic_id argument, so we add the attribute topic.id to the URL 
template tag. Now each topic in the list of topics is a link to a topic page, 
such as Attp://localhost:8000/topics/1/. 

When you refresh the topics page and click a topic, you should see a 
page that looks like Figure 18-5. 


There's a subtle but important difference between topic.id and topic id. The expres- 
sion topic.id examines a topic and retrieves the value of the corresponding ID. The 
variable topic id is a reference to that ID in the code. If you run into errors when 
working with IDs, make sure yowre using these expressions in the appropriate ways. 


Getting Started with Django 399 


400 


eee D < ® localhost:8000/topics/1/ e oO + 
localhost:8000/topics/1/ 
Topic: Chess 
Entries: 
* May 21,2022 05:04 


In the opening phase of the game, it's important to bring out your bishops and knights. These pieces are powerful and 
maneuverable enough to play a significant role in the beginning moves of a game. 


May 21, 2022 05:04 


The opening is the first part of the game, roughly the first ten moves or so. In the opening, it's a good idea to do three 
things — bring out your bishops and knights, try to control the center of the board, and castle your king. 


Of course, these are just guidelines. It will be important to learn when to follow these guidelines and when to 
disregard these suggestions. 


Figure 18-5: The detail page for a single topic, showing all entries for a topic 


TRY IT YOURSELF 


18-7. Template Documentation: Skim the Django template documentation at 
https;//docs.djangoproject.com/en/A. I/ref/templates. You can refer back to it 
when yov're working on your own projects. 

18-8. Pizzeria Pages: Add a page to the Pizzeria project from Exercise 18-6 
(page 392) that shows the names of available pizzas. Then link each pizza 
name to a page displaying the pizza's toppings. Make sure you use template 


inheritance to build your pages efficiently. 


Summary 


Chapter 18 


In this chapter, you learned how to start building a simple web app using 
the Django framework. You saw a brief project specification, installed 
Django to a virtual environment, set up a project, and checked that the 
project was set up correctly. You set up an app and defined models to rep- 
resent the data for your app. You learned about databases and how Django 
helps you migrate your database after you make a change to your models. 
You created a superuser for the admin site, and you used the admin site to 
enter some initial data. 

You also explored the Django shell, which allows you to work with your 
project's data in a terminal session. You learned how to define URLs, cre- 
ate view functions, and write templates to make pages for your site. You also 
used template inheritance to simplify the structure of individual templates 
and make it easier to modify the site as the project evolves. 


In Chapter 19, you'll make intuitive, user-friendly pages that allow 
users to add new topics and entries and edit existing entries without going 
through the admin site. You'll also add a user registration system, allow- 
ing users to create an account and make their own learning log. This is the 
heart of a web app—the ability to create something that any number of 
users can interact with. 
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USER ACCOUNTS 


At the heart of a web application is the abil- 
ity for any user, anywhere in the world, to 


register an account with your app and start 

using it. In this chapter, you’ll build forms so 
users can add their own topics and entries, and edit 
existing entries. You'll also learn how Django guards 
against common attacks against form-based pages, so 
you won't have to spend much time thinking about 
securing your apps. 


You'll also implement a user authentication system. You'll build a regis- 
tration page for users to create accounts, and then restrict access to certain 
pages to logged-in users only. Then you'll modify some of the view func- 
tions so users can only see their own data. You'll learn to keep your users' 
data safe and secure. 


404 
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Before we build an authentication system for creating accounts, we’ll first 
add some pages that allow users to enter their own data. We'll give users the 
ability to add a new topic, add a new entry, and edit their previous entries. 

Currently, only a superuser can enter data through the admin site. We 
don’t want users to interact with the admin site, so we'll use Django's form- 
building tools to build pages that allow users to enter data. 


Adding New Topics 


Let's start by allowing users to add a new topic. Adding a form-based page 
works in much the same way as adding the pages we've already built: we 
define a URL, write a view function, and write a template. The one signifi- 
cant difference is the addition of a new module called forms.py, which will 
contain the forms. 


The Topic ModelForm 


Any page that lets a user enter and submit information on a web page involves 
an HTML element called a form. When users enter information, we need to 
validate that the information provided is the right kind of data and is not mali- 
cious, such as code designed to interrupt our server. We then need to process 
and save valid information to the appropriate place in the database. Django 
automates much of this work. 

The simplest way to build a form in Django is to use a ModelForm, which 
uses the information from the models we defined in Chapter 18 to build a 
form automatically. Write your first form in the file forms.py, which should 
be created in the same directory as models.py: 


from django import forms 


from .models import Topic 


© class TopicForm(forms.ModelForm): 


class Meta: 
model = Topic 
fields = ['text'] 
labels = ('text': ''} 


We first import the forms module and the model we'll work with, 
Topic. We then define a class called TopicForm, which inherits from forms 
.ModelForm 6. 

The simplest version of a ModelForm consists of a nested Meta class telling 
Django which model to base the form on and which fields to include in the 
form. Here we specify that the form should be based on the Topic model 6, 
and that it should only include the text field ©. The empty string in the 
labels dictionary tells Django not to generate a label for the text field O. 


The new_topic URL 


The URL for a new page should be short and descriptive. When the 
user wants to add a new topic, we'll send them to Attp://localhost:8000/ 
new. topic/. Here's the URL pattern for the new topic page; add this to 
learning. logs/urls.py: 


learning logs/ 
urls.py 


# Page for adding a new topic. 
path('new topic/', views.new topic, name-'new topic'), 


This URL pattern sends requests to the view function new topic(), which 
we'll write next. 


The new topic() View Function 


The new topic() function needs to handle two different situations: ini- 
tial requests for the new topic page, in which case it should show a blank 
form; and the processing of any data submitted in the form. After data 
from a submitted form is processed, it needs to redirect the user back to 
the topics page: 


views.py from django.shortcuts import render, redirect 
from .forms import TopicForm 


def new_topic(request): 
"""Add a new topic. 
e if request.method !- 'POST': 
# No data submitted; create a blank form. 
e form - TopicForm() 
else: 
# POST data submitted; process data. 


e form = TopicForm(data-request.POST) 

(4) if form.is_valid(): 

e form.save() 

e return redirect('learning logs:topics') 
# Display a blank or invalid form. 

[7) context - ('form': form) 


return render(request, 'learning logs/new topic.html', context) 


We import the function redirect, which we'll use to redirect the user 
back to the topics page after they submit their topic. We also import the 
form we just wrote, TopicForm. 
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GET and POST Requests 


The two main types of requests you'll use when building apps are GET and 
POST. You use GET requests for pages that only read data from the server. 
You usually use POST requests when the user needs to submit information 
through a form. We'll be specifying the POST method for processing all of 
our forms. (A few other kinds of requests exist, but we won't use them in 
this project.) 

The new topic() function takes in the request object as a parameter. 
When the user initially requests this page, their browser will send a GET 
request. Once the user has filled out and submitted the form, their browser 
will submit a POST request. Depending on the request, we'll know whether 
the user is requesting a blank form (GET) or asking us to process a com- 
pleted form (POST). 

We use an if test to determine whether the request method is GET or 
POST @. If the request method isn't POST, the request is probably GET, so 
we need to return a blank form. (If it’s another kind of request, it's still safe 
to return a blank form.) We make an instance of TopicForm @, assign it to 
the variable form, and send the form to the template in the context diction- 
ary 9. Because we included no arguments when instantiating TopicForm, 
Django creates a blank form that the user can fill out. 

If the request method is POST, the else block runs and processes the 
data submitted in the form. We make an instance of TopicForm 6 and pass 
it the data entered by the user, which is assigned to request.POST. The form 
object that's returned contains the information submitted by the user. 

We can't save the submitted information in the database until we've 
checked that it's valid O. The is valid() method checks that all required 
fields have been filled in (all fields in a form are required by default) and 
that the data entered matches the field types expected—for example, that 
the length of text is less than 200 characters, as we specified in models.py in 
Chapter 18. This automatic validation saves us a lot of work. If everything 
is valid, we can call save() 9, which writes the data from the form to the 
database. 

Once we've saved the data, we can leave this page. The redirect() func- 
tion takes in the name of a view and redirects the user to the page associated 
with that view. Here we use redirect() to redirect the user's browser to the 
topics page @, where the user should see the topic they just entered in the list 
of topics. 

The context variable is defined at the end of the view function, and the 
page is rendered using the template new fopic.html, which we'll create next. 
This code is placed outside of any if block; it will run if a blank form was 
created, and it will run if a submitted form is determined to be invalid. An 
invalid form will include some default error messages to help the user sub- 
mit acceptable data. 


new. fopic. html 


topics.html 


The new topic Template 


Now we'll make a new template called new. topic.html to display the form we 
just created: 


{% extends "learning logs/base.html" %} 


{% block content %} 
<p>Add a new topic:</p> 


«form action="{% url 'learning logs:new topic' %}" method='post'> 
{% csrf_token %} 
{{ form.as div }} 
<button name="submit">Add topic</button> 

</form> 


{% endblock content %} 


This template extends base.himl, so it has the same base structure as the 
rest of the pages in Learning Log. We use the <form></form> tags to define 
an HTML form @. The action argument tells the browser where to send the 
data submitted in the form; in this case, we send it back to the view function 
new_topic(). The method argument tells the browser to submit the data as a 
POST request. 

Django uses the template tag {% csrf_token %} @ to prevent attackers 
from using the form to gain unauthorized access to the server. (This kind 
of attack is called a cross-site request forgery.) Next, we display the form; here 
you can see how simple Django can make certain tasks, such as displaying 
a form. We only need to include the template variable {{ form.as_div }} 
for Django to create all the fields necessary to display the form automati- 
cally ©. The as div modifier tells Django to render all the form elements as 
HTML <div></div> elements; this is a simple way to display the form neatly. 

Django doesn't create a submit button for forms, so we define one before 
closing the form O. 


Linking to the new topic Page 


Next, we include a link to the new topic page on the topics page: 


«a href="{% url 'learning logs:new topic' %}">Add a new topic</a> 
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Place the link after the list of existing topics. Figure 19-1 shows the 
resulting form; try using the form to add a few new topics of your own. 


e < È localhost:B000/new. topic) ( UE. 


Learning Log - Topics 


Add a new topic: 


Add topic 


Figure 19-1: The page for adding a new topic 


Adding New Entries 


Now that the user can add a new topic, they'll want to add new entries too. 
We'll again define a URL, write a view function and a template, and link to 
the page. But first, we'll add another class to forms.py. 


The Entry ModelForm 


We need to create a form associated with the Entry model, but this time, 
with a bit more customization than TopicForm: 


rorm 


from .models import Topic, Entry 


class EntryForm(forms.ModelForm): 
class Meta: 
model - Entry 
fields - ['text'] 
labels - ('text': '') 
widgets = ('text': forms.Textarea(attrs-('cols': 80})} 


We update the import statement to include Entry as well as Topic. We 
make a new class called EntryForm that inherits from forms.ModelForm. The 
EntryForm class has a nested Meta class listing the model it's based on, and the 
field to include in the form. We again give the field 'text' a blank label 6. 

For EntryForn, we include the widgets attribute @. A widget isan HTML 
form element, such as a single-line text box, multiline text area, or drop- 
down list. By including the widgets attribute, you can override Django's 


learning_logs/ 
urls. py 


views. py 
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default widget choices. Here we're telling Django to use a forms.Textarea ele- 
ment with a width of 80 columns, instead of the default 40 columns. This 
gives users enough room to write a meaningful entry. 


The new entry URL 


New entries must be associated with a particular topic, so we need to include 
a topic id argument in the URL for adding a new entry. Here's the URL, 
which you add to learning. logs/urls.py: 


# Page for adding a new entry. 
path('new entry/«int:topic id»/', views.new entry, name-'new entry'), 


This URL pattern matches any URL with the form Attp://localhost:8000/ 
new. entry/id/, where id is a number matching the topic ID. The code 
«int:topic id» captures a numerical value and assigns it to the variable 
topic id. When a URL matching this pattern is requested, Django sends the 
request and the topic's ID to the new entry() view function. 


The new_entry() View Function 


The view function for new_entry is much like the function for adding a new 
topic. Add the following code to your views.py file: 


from .forms import TopicForm, EntryForm 


def new entry(request, topic id): 
"""Add a new entry for a particular topic. 
topic - Topic.objects.get(id-topic id) 


if request.method !- 'POST': 
# No data submitted; create a blank form. 
form - EntryForm() 
else: 
# POST data submitted; process data. 
form = EntryForm(data-request.POST) 
if form.is valid(): 
new entry - form.save(commit-False) 
new entry.topic - topic 
new entry.save() 
return redirect('learning logs:topic', topic id-topic id) 


# Display a blank or invalid form. 
context - ('topic': topic, 'form': form] 
return render(request, 'learning logs/new entry.html', context) 
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We update the import statement to include the EntryForm we just made. 
The definition of new_entry() has a topic_id parameter to store the value it 
receives from the URL. We’ll need the topic to render the page and process 
the form’s data, so we use topic id to get the correct topic object 6. 

Next, we check whether the request method is POST or GET 0. The 
if block executes if it’s a GET request, and we create a blank instance of 
EntryForm ®. 

If the request method is POST, we process the data by making an instance 
of EntryForm, populated with the POST data from the request object O. We 
then check whether the form is valid. If it is, we need to set the entry object's 
topic attribute before saving it to the database. When we call save(), we include 
the argument commit-False 9 to tell Django to create a new entry object and 
assign it to new entry, without saving it to the database yet. We set the topic 
attribute of new entry to the topic we pulled from the database at the begin- 
ning of the function ©. Then we call save() with no arguments, saving the 
entry to the database with the correct associated topic. 

The redirect() call requires two arguments: the name of the view we 
want to redirect to and the argument that view function requires 9. Here, 
we're redirecting to topic(), which needs the argument topic id. This view 
then renders the topic page that the user made an entry for, and they 
should see their new entry in the list of entries. 

At the end of the function, we create a context dictionary and render the 
page using the new entry.html template. This code will execute for a blank 
form, or for a form that's been submitted but turns out to be invalid. 


The new entry Template 


As you can see in the following code, the template for new entry is similar to 
the template for new topic: 


new entry.html — (5 extends "learning logs/base.html" %} 
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{% block content 4) 
@ <p><a href="{% url ‘learning logs:topic' topic.id %}">{{ topic }}</a></p> 


<p>Add a new entry:</p> 
@ <form action="{% url ‘learning logs:new entry' topic.id X)" method='post'> 
{% csrf_token %} 
{{ form.as div }} 
«button name-'submit'»Add entry</button> 
«/form» 


{% endblock content %} 


We show the topic at the top of the page @, so the user can see which 
topic they're adding an entry to. The topic also acts as a link back to the 
main page for that topic. 
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topic. html 


The form’s action argument includes the topic.id value in the URL, 
so the view function can associate the new entry with the correct topic @. 
Other than that, this template looks just like new_topic.himl. 


Linking to the new entry Page 


Next, we need to include a link to the new entry page from each topic page, 
in the topic template: 


{% extends "lez 


(4 block content %} 


<p>Topic: {{ topic }}</p> 


<p>Entries:</p> 
<p> 

<a href="{% url ‘learning logs:new entry' topic.id %}">Add new entry</a> 
</p> 


{% endblock content %} 


We place the link to add entries just before showing the entries, 
because adding a new entry will be the most common action on this page. 
Figure 19-2 shows the new_entry page. Now users can add new topics and as 
many entries as they want for each topic. Try out the new_entry page by add- 
ing a few entries to some of the topics you've created. 


eee mq) < & localhost:8000/new_entry/1/ è © Ô + © 


localhost:8000/new_entry/1/ 


Learning Log - Topics 
Chess 


Add a new entry: 


The bishops and knights are good pieces to have out in the opening phase of the game. They're both 


powerful enough to be useful in attacking your opponent, but not so powerful that you can't afford to lose 
them in an early trade. 


Add entry 


Figure 19-2: The new_entry page 
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Editing Entries 


Now we'll make a page so users can edit the entries they've added. 


The edit entry URL 


The URL for the page needs to pass the ID of the entry to be edited. Here's 
learning. logs/urls.py: 


urls.py 


# Page for editing an entry. 
path('edit entry/«int:entry id»/', views.edit entry, name-'edit entry'), 


This URL pattern matches URLs like hitp://locathost:8000/edit_entry/id/. 
Here the value of id is assigned to the parameter entry id. Django sends 
requests that match this format to the view function edit entry(). 


The edit entry() View Function 


When the edit entry page receives a GET request, the edit entry() func- 
tion returns a form for editing the entry. When the page receives a POST 
request with revised entry text, it saves the modified text into the database: 


views.py 


from .models import Topic, Entry 


def edit entry(request, entry id): 
"""Edit an existing entry.""" 
e entry = Entry.objects.get(id-entry id) 
topic - entry.topic 


if request.method !- 'POST': 
# Initial request; pre-fill form with the current entry. 


e form - EntryForm(instance-entry) 
else: 
# POST data submitted; process data. 
e form - EntryForm(instance-entry, data-request.POST) 


if form.is valid(): 
form.save() 
return redirect('learning logs:topic', topic id-topic.id) 


oo 


context = ('entry': entry, 'topic': topic, 'form': form} 
return render(request, 'learning logs/edit entry.html', context) 


We first import the Entry model. We then get the entry object that the 
user wants to edit 6 and the topic associated with this entry. In the if 
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block, which runs for a GET request, we make an instance of EntryForm with 
the argument instance-entry @. This argument tells Django to create the 
form, prefilled with information from the existing entry object. The user 
will see their existing data and be able to edit that data. 

When processing a POST request, we pass both the instance-entry and 
the data=request.POST arguments ®. These arguments tell Django to create a 
form instance based on the information associated with the existing entry 
object, updated with any relevant data from request.POST. We then check 
whether the form is valid; if it is, we call save() with no arguments because the 
entry is already associated with the correct topic O. We then redirect to the 
topic page, where the user should see the updated version of the entry they 
edited 9. 

If we're showing an initial form for editing the entry or if the submitted 
form is invalid, we create the context dictionary and render the page using 
the edit, entry.html template. 


The edit entry Template 


Next, we create an edit. entry.html template, which is similar to new entry.html: 


edit entry.html ^ (X extends "learning logs/base.html" %} 
{% block content X) 
<p><a href="{% url 'learning logs:topic' topic.id %}">{{ topic }}</a></p> 
<p>Edit entry:</p> 
€ <form action="{% url 'learning logs:edit entry' entry.id Xj" method-'post'» 
(4 csrf token %} 
(( form.as div }} 
e «button name-"submit"»Save changes</button> 


</form> 


{% endblock content %} 


The action argument sends the form back to the edit_entry() function 
for processing 6. We include the entry.id as an argument in the (X url %} 
tag, so the view function can modify the correct entry object. We label the 
submit button as Save changes to remind the user they're saving edits, not 
creating a new entry @. 


Linking to the edit entry Page 


Now we need to include a link to the edit entry page for each entry on the 
topic page: 


fopic.html 
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<p>{{ entry.date added|date:'M d, Y H:i' }}</p> 
<p>{{ entry.text|linebreaks }}</p> 
<p> 
«a href="{% url ‘learning logs:edit entry' entry.id %}"> 
Edit entry</a></p> 
</li> 
--snip-- 


We include the edit link after each entry’s date and text has been dis- 
played. We use the {% url %} template tag to determine the URL for the 
named URL pattern edit_entry, along with the ID attribute of the current 
entry in the loop (entry.id). The link text Edit entry appears after each entry 
on the page. Figure 19-3 shows what the topic page looks like with these 
links. 


eee D < ® localhost:8000/topics/1 e © Â + 


localhost:8000/topics/1 
Learning Log - Topics 
Topic: Chess 
Entries: 
Add new entry 
* May 20, 2022 21:21 


The bishops and knights are good pieces to have out in the opening phase of the game. They're both powerful 
enough to be useful in attacking your opponent, but not so powerful that you can't afford to lose them in an 
early trade. 


Edit entry 


May 20, 2022 03:44 


In the opening phase of the game, it’s important to bring out your bishops and knights. These pieces are 
powerful and maneuverable enough to play a significant role in the beginning moves of a game. 


Edit entry 


May 20, 2022 03:43 


The opening is the first part of the game, roughly the first ten moves or so. In the opening, it’s a good idea to do 
three things — bring out your bishops and knights, try to control the center of the board, and castle your king. 


Figure 19-3: Each entry now has a link for editing that entry. 


Learning Log now has most of the functionality it needs. Users can add 
topics and entries, and they can read through any set of entries they want. 
In the next section, we’ll implement a user registration system so anyone 
can make an account with Learning Log and create their own set of topics 
and entries. 
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TRY IT YOURSELF 


19-1. Blog: Start a new Django project called Blog. Create an app called 
blogs, with one model that represents an overall blog, and one model that rep- 
resents an individual blog post. Give each model an appropriate set of fields. 


Create a superuser for the project, and use the admin site to make a blog and 
a couple of short posts. Make a home page that shows all posts in an appropri- 
ate order. 

Create pages for making a blog, for making new posts, and for editing 
existing posts. Use your pages to make sure they work. 


Setting Up User Accounts 


In this section, we’ll set up a user registration and authorization system so 
people can register an account, log in, and log out. We'll create a new app 
to contain all the functionality related to working with users. We’ll use the 
default user authentication system included with Django to do as much 
of the work as possible. We'll also modify the Topic model slightly so every 
topic belongs to a certain user. 


The accounts App 


We'll start by creating a new app called accounts, using the startapp command: 


(1l env)learning log$ python manage.py startapp accounts 
(11 env)learning log$ 1s 


@ accounts db.sqlite3 learning logs ll env ll project manage.py 


(1l env)learning log$ ls accounts 


€ init .py admin.py apps.py migrations models.py tests.py views.py 


settings.py 


The default authentication system is built around the concept of user 
accounts, so using the name accounts makes integration with the default sys- 
tem easier. The startapp command shown here makes a new directory called 
accounts ® with a structure identical to the learning logs app 6. 


Adding accounts to settings.py 
We need to add our new app to INSTALLED APPS in settings.py, like so: 


'accounts' , 
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Il project/urls.py 


accounts/urls.py 
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Now Django will include the accounts app in the overall project. 


Including the URLs from accounts 


Next, we need to modify the root urls.py so it includes the URLs we'll write 
for the accounts app: 


path( 'accounts/', include('accounts.urls')), 


We add a line to include the file urls.py from accounts. This line will 
match any URL that starts with the word accounts, such as http://localhost: 
8000/accounts/login/. 


The Login Page 


We'll first implement a login page. We'll use the default login view Django 
provides, so the URL pattern for this app looks a little different. Make a 
new urls.py file in the directory ll_project/accounts/ and add the following 
to it: 


Defines URL patterns for accounts. 
from django.urls import path, include 


app name - 'accounts' 
urlpatterns - [ 
# Include default auth urls. 
path('', include('django.contrib.auth.urls')), 


We import the path function, and then import the include function so 
we can include some default authentication URLs that Django has defined. 
These default URLs include named URL patterns, such as 'login' and 
'logout'. We set the variable app name to 'accounts' so Django can distin- 
guish these URLs from URLs belonging to other apps. Even default URLs 
provided by Django, when included in the accounts app’s urls.py file, will be 
accessible through the accounts namespace. 


login.html 


The login page's pattern matches the URL hitp://locathost:8000/accounts/ 
login/. When Django reads this URL, the word accounts tells Django to 
look in accounts/urls.py, and login tells it to send requests to Django’s 
default login view. 


The login Template 


When the user requests the login page, Django will use a default view 
function, but we still need to provide a template for the page. The default 
authentication views look for templates inside a folder called registration, so 
we'll need to make that folder. Inside the U_project/accounts/ directory, make 

a directory called templates; inside that, make another directory called registra- 
tion. Here's the login.html template, which should be saved in ll_project/accounts/ 
templates/registration: 


{% extends ‘learning logs/base.html' X) 
{% block content %} 
{% if form.errors %} 
<p>Your username and password didn't match. Please try again.</p> 
{% endif %} 
«form action="{% url 'accounts:login' %}" method-'post'» 
(4 csrf token %} 
(( form.as div }} 


«button name-"submit"»Log in«/button» 
</form> 


{% endblock content %} 


This template extends base.himl to ensure that the login page will have 
the same look and feel as the rest of the site. Note that a template in one 
app can inherit from a template in another app. 

If the form’s errors attribute is set, we display an error message 6, report- 
ing that the username and password combination doesn't match anything 
stored in the database. 

We want the login view to process the form, so we set the action argu- 
ment as the URL of the login page @. The login view sends a form object 
to the template, and it's up to us to display the form 6 and add a submit 
button O. 


The LOGIN REDIRECT URL Settting 


Once a user logs in successfully, Django needs to know where to send that 
user. We control this in the settings file. 
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settings.py 


base.html 
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oo 


Add the following code to the end of settings.py: 


# My settings. 
LOGIN REDIRECT URL - 'learning logs:index' 


With all the default settings in settings.py, it's helpful to mark off the 
section where we're adding new settings. The first new setting we'll add is 
LOGIN REDIRECT URL, which tells Django which URL to redirect to after a suc- 
cessful login attempt. 


Linking to the Login Page 


Let's add the login link to base.himl so it appears on every page. We don't 
want the link to display when the user is already logged in, so we nest it 
inside an (4 if X) tag: 


«a href="{% url ‘learning logs:index' %}">Learning Log< 
«a href="{% url ‘learning logs:topics' %}">Topics</a> - 
{% if user.is authenticated %} 

Hello, {{ user.username }}. 
{% else %} 

<a href="{% url 'accounts:login' %}">Log in</a> 
{% endif %} 


In Django’s authentication system, every template has a user object 
available that always has an is_authenticated attribute set: the attribute is 
True if the user is logged in and False if they aren’t. This attribute allows you 
to display one message to authenticated users and another to unauthenti- 
cated users. 

Here we display a greeting to users currently logged in 6. Authenticated 
users have an additional username attribute set, which we use to personalize 
the greeting and remind the user they're logged in @. For users who haven't 
been authenticated, we display a link to the login page 9. 


Using the Login Page 

We've already set up a user account, so let's log in to see if the page works. 
Go to Attp://localhost:8000/admin/. If yowte still logged in as an admin, look 
for a logout link in the header and click it. 

When youre logged out, go to http://localhost:8000/accounts/login/. You 
should see a login page similar to the one shown in Figure 19-4. Enter the 
username and password you set up earlier, and you should be brought back 
to the home page. The header on the home page should display a greeting 
personalized with your username. 


base. html 


ecg < E localhost:8000/accounts/logiey 


Learning Log - Topics - Log in 


Username: i| admin 
Password: eeeseees 


Figure 19-4: The login page 


Logging Out 

Now we need to provide a way for users to log out. Logout requests should 
be submitted as POST requests, so we'll add a small logout form to base.himl. 
When users click the logout button, they'll go to a page confirming that 
they've been logged out. 


Adding a Logout Form to base.html 


We'll add the form for logging out to base.himl so it's available on every 
page. We'll include it in another if block, so only users who are already 
logged in can see it: 


(4 if user.is authenticated %} 
«hr /» 
«form action="{% url 'accounts:logout' Xj" method-'post'» 
(4 csrf token %} 
«button name-'submit'»Log out</button> 
«/form» 
{% endif %} 


The default URL pattern for logging out is 'accounts/logout/'. However, 
the request has to be sent as a POST request; otherwise, attackers can eas- 
ily force logout requests. To make the logout request use POST, we define a 
simple form. 

We place the form at the bottom of the page, below a horizontal rule 
element («hr /») ®. This is an easy way to always keep the logout button in a 
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settings.py 


accounts/urls.py 
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consistent position below any other content on the page. The form itself has 
the logout URL as its action argument, and 'post' as the request method 6. 
Every form in Django needs to include the {% csrf_token %}, even a simple 
form like this one. This form is empty except for the submit button. 


The LOGOUT_REDIRECT_URL Setting 


When the user clicks the logout button, Django needs to know where to 
send them. We control this behavior in settings.py: 


LOGOUT REDIRECT URL = 'learning logs:index' 


The LOGOUT REDIRECT URL setting shown here tells Django to redirect 
logged-out users back to the home page. This is a simple way to confirm 
that they were logged out, because they should no longer see their user- 
name after logging out. 


The Registration Page 


Next, we'll build a page so new users can register. We'll use Django's default 
UserCreationForm, but write our own view function and template. 


The register URL 


The following code provides the URL pattern for the registration page, 
which should be placed in accounts/urls.py: 


from . import views 


# Registration page. 
path('register/', views.register, name='register'), 


We import the views module from accounts, which we need because 
we're writing our own view for the registration page. The pattern for the 
registration page matches the URL http://localhost:8000/accounts/register/ and 
sends requests to the register() function we’re about to write. 


The register() View Function 


The register() view function needs to display a blank registration form 
when the registration page is first requested, and then process completed 
registration forms when they’re submitted. When a registration is success- 
ful, the function also needs to log the new user in. Add the following code 
to accounts/views.py: 


accounis/ from django.shortcuts import render, redirect 
views.py from django.contrib.auth import login 
from django.contrib.auth.forms import UserCreationForm 


def register(request): 
"""Register a new user. 
if request.method !- 'POST': 
# Display blank registration form. 
o form = UserCreationForm() 
else: 
# Process completed form. 
form = UserCreationForm(data=request .POST) 


if form.is_valid(): 
new_user = form.save() 
# Log the user in and then redirect to home page. 
login(request, new_user) 
return redirect('learning logs:index') 


oo oco ® 


# Display a blank or invalid form. 
context - ('form': form) 
return render(request, 'registration/register.html', context) 


We import the render() and redirect() functions, and then we import 
the login() function to log the user in if their registration information is 
correct. We also import the default UserCreationForm. In the register() func- 
tion, we check whether we're responding to a POST request. If we're not, we 
make an instance of UserCreationForm with no initial data 6. 

If we're responding to a POST request, we make an instance of 
UserCreationForm based on the submitted data 6. We check that the data 
is valid 6 —in this case, that the username has the appropriate characters, 
the passwords match, and the user isn't trying to do anything malicious in 
their submission. 

If the submitted data is valid, we call the form's save() method to save 
the username and the hash of the password to the database O. The save() 
method returns the newly created user object, which we assign to new user. 
When the user's information is saved, we log them in by calling the login() 
function with the request and new user objects 6, which creates a valid ses- 
sion for the new user. Finally, we redirect the user to the home page ©, 


User Accounts 421 


422 


register.html 


base. html 


Chapter 19 


where a personalized greeting in the header tells them their registration 
was successful. 

At the end of the function, we render the page, which will be either a 
blank form or a submitted form that’s invalid. 


The register Template 


Now create a template for the registration page, which will be similar to the 
login page. Be sure to save it in the same directory as login. himl: 


{% extends "learning logs/base.html" %} 
{% block content %} 
<form action="{% url 'accounts:register' %}" method='post'> 
{% csrf_token %} 
{{ form.as div }} 


<button name="submit">Register</button> 
</form> 


{% endblock content %} 


This should look like the other form-based templates we’ve been writ- 
ing. We use the as_div method again so Django will display all the fields in 
the form appropriately, including any error messages if the form isn’t filled 
out correctly. 


Linking to the Registration Page 


Next, we'll add code to show the registration page link to any user who isn't 
currently logged in: 


<a href="{% url 'accounts:register' %}">Register</a> - 


f 


Now users who are logged in see a personalized greeting and a logout 
button. Users who aren't logged in see a registration link and a login link. 
Try out the registration page by making several user accounts with different 
usernames. 

In the next section, we'll restrict some of the pages so they're available 
only to registered users, and we'll make sure every topic belongs to a spe- 
cific user. 


The registration system we've set up allows anyone to make any number of accounts 
for Learning Log. Some systems require users to confirm their identity by sending a 
confirmation email that users must reply to. By doing so, the system generates fewer 
spam accounts than the simple system we're using here. However, when you're learn- 
ing to build apps, it’s perfectly appropriate to practice with a simple user registration 
system like the one we're using. 


TRY IT YOURSELF 


19-2. Blog Accounts: Add a user authentication and registration system to the 


Blog project you started in Exercise 19-1 (page 415). Make sure logged-in users 
see their username somewhere on the screen and unregistered users see a link 


to the registration page. 


Allowing Users to Own Their Data 


learning_logs/ 
views. py 


Users should be able to enter private data in their learning logs, so we'll 
create a system to figure out which data belongs to which user. Then we'll 
restrict access to certain pages so users can only work with their own data. 

We'll modify the Topic model so every topic belongs to a specific user. 
This will also take care of entries, because every entry belongs to a specific 
topic. We'll start by restricting access to certain pages. 


Restricting Access with &login required 


Django makes it easy to restrict access to certain pages through the login 
. required decorator. Recall from Chapter 11 that a decorator is a directive 
placed just before a function definition, which modifies how the function 
behaves. Let's look at an example. 


Restricting Access to the Topics Page 


Each topic will be owned by a user, so only registered users can request the 
topics page. Add the following code to learning. logs/views.py: 


from django.contrib.auth.decorators import login required 


Qlogin required 
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We first import the login required() function. We apply login required() 
as a decorator to the topics() view function by prepending login required 
with the 0 symbol. As a result, Python knows to run the code in login 
_required() before the code in topics(). 

The code in login required() checks whether a user is logged in, and 
Django runs the code in topics() only if they are. If the user isn't logged in, 
they're redirected to the login page. 

To make this redirect work, we need to modify settings.py so Django knows 
where to find the login page. Add the following at the end of settings.py: 


p ) ( ) - sarni G q r 
R E UR earning logs:index 
Jl 'EDIRE( ) ‘learning s:index 


LOGIN URL = ‘accounts: login' 


Now when an unauthenticated user requests a page protected by the 
@login_required decorator, Django will send the user to the URL defined by 
LOGIN URL in settings.py. 

You can test this setting by logging out of any user accounts and going 
to the home page. Click the Topics link, which should redirect you to the 
login page. Then log in to any of your accounts, and from the home page, 
click the Topics link again. You should be able to access the topics page. 


Restricting Access Throughout Learning Log 


Django makes it easy to restrict access to pages, but you have to decide which 
pages to protect. It's best to think about which pages need to be unrestricted 
first, and then restrict all the other pages in the project. You can easily cor- 
rect over-restricted access, and it's less dangerous than leaving sensitive 
pages unrestricted. 

In Learning Log, we'll keep the home page and the registration page 
unrestricted. We'll restrict access to every other page. 

Here's learning. logs/views.py with 61ogin required decorators applied to 
every view except index(): 


@login_ required 


@login required 


models. py 


@login_ required 


@login required 


request en 
JN or £ 


Try accessing each of these pages while logged out; you should be redi- 
rected back to the login page. You'll also be unable to click links to pages 
such as new topic. But if you enter the URL hitp://localhost:8 000/new_topic/, 
you'll be redirected to the login page. You should restrict access to any URL 
that's publicly accessible and relates to private user data. 


Connecting Data to Certain Users 


Next, we need to connect the data to the user who submitted it. We only need 
to connect the data highest in the hierarchy to a user, and the lower-level data 
will follow. In Learning Log, topics are the highest level of data in the app, 
and all entries are connected to a topic. As long as each topic belongs to a 
specific user, we can trace the ownership of each entry in the database. 
We'll modify the Topic model by adding a foreign key relationship to a 
user. We'll then have to migrate the database. Finally, we'll modify some of the 
views so they only show the data associated with the currently logged-in user. 


Modifying the Topic Model 


The modification to models.py is just two lines: 


from django.contrib.auth.models import User 


owner - models.ForeignKey(User, on delete-models.CASCADE) 


We import the User model from django.contrib.auth. Then we add an 
owner field to Topic, which establishes a foreign key relationship to the User 
model. If a user is deleted, all the topics associated with that user will be 
deleted as well. 
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Identifying Existing Users 
When we migrate the database, Django will modify the database so it can 
store a connection between each topic and a user. To make the migration, 
Django needs to know which user to associate with each existing topic. The 
simplest approach is to start by assigning all existing topics to one user—for 
example, the superuser. But first, we need to know that user’s ID. 

Let’s look at the IDs of all users created so far. Start a Django shell ses- 
sion and issue the following commands: 


(11_env)learning log$ python manage.py shell 
@ >>> from django.contrib.auth.models import User 
@ >>> User.objects.all() 

<QuerySet [<User: ll admin», «User: eric», «User: willie>]> 
© >>> for user in User.objects.all(): 

print(user.username, user.id) 

1l admin 1 

eric 2 

willie 3 

>>> 


We first import the User model into the shell session ®. We then look at 
all the users that have been created so far @. The output shows three users 
for my version of the project: 11 admin, eric, and willie. 

Next, we loop through the list of users and print each user’s username 
and ID ©. When Django asks which user to associate the existing topics 
with, we'll use one of these ID values. 


Migrating the Database 


Now that we know the IDs, we can migrate the database. When we do this, 
Python will ask us to connect the Topic model to a particular owner tempo- 
rarily or to add a default to our models.py file to tell it what to do. Choose 
option 1: 


© (11 env)learning log$ python manage.py makemigrations learning logs 
@ It is impossible to add a non-nullable field ‘owner’ to topic without 
specifying a default. This is because... 
© Please select a fix: 
1) Provide a one-off default now (will be set on all existing rows with a 
null value for this column) 
2) Quit and manually define a default value in models.py. 
O Select an option: 1 
© Please enter the default value now, as valid Python 
The datetime and django.utils.timezone modules are available... 
Type 'exit' to exit this prompt 
9 »»1 
Migrations for 'learning logs': 
learning logs/migrations/0003 topic owner.py 
- Add field owner to topic 
(1l env)learning log$ 


We start by issuing the makemigrations command @. In the output, 
Django indicates that we’re trying to add a required (non-nullable) field to 
an existing model (topic) with no default value specified @. Django gives 
us two options: we can provide a default right now, or we can quit and add 
a default value in models.py 9. Here I've chosen the first option @. Django 
then asks us to enter the default value 9. 

To associate all existing topics with the original admin user, 11 admin, I 
entered the user ID of 1 O. You can use the ID of any user you've created; 
it doesn't have to be a superuser. Django then migrates the database using 
this value and generates the migration file 0003 topic owner.py, which adds 
the field owner to the Topic model. 

Now we can execute the migration. Enter the following in an active vir- 
tual environment: 


(1l env)learning log$ python manage.py migrate 
Operations to perform: 

Apply all migrations: admin, auth, contenttypes, learning logs, sessions 
Running migrations: 


€ Applying learning logs.0003 topic owner... OK 


(1l env)learning log$ 


Django applies the new migration, and the result is 0K 6). 
We can verify that the migration worked as expected in a shell session, 
like this: 


»»» from learning logs.models import Topic 
»»» for topic in Topic.objects.all(): 
print(topic, topic.owner) 


Chess 11 admin 


Rock Climbing 1l admin 
>>> 


We import Topic from learning_logs.models and then loop through all 
existing topics, printing each topic and the user it belongs to. You can see 
that each topic now belongs to the user 11 admin. (If you get an error when 
you run this code, try exiting the shell and starting a new shell.) 


You can simply reset the database instead of migrating, but that will lose all existing 
data. It’s good practice to learn how to migrate a database while maintaining the 
integrity of users’ data. If you do want to start with a fresh database, issue the com- 
mand python manage.py flush to rebuild the database structure. You'll have to create 
a new superuser, and all of your data will be gone. 


Restricting Topics Access to Appropriate Users 


Currently, if you're logged in, you'll be able to see all the topics, no matter 
which user you're logged in as. We'll change that by showing users only the 
topics that belong to them. 
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Make the following change to the topics() function in views.py: 


learning_logs/ 
views. py D1 OB in T equi red 


topics = Topic.objects.filter(owner-request.user) .order by('date added') 


X1 


LOD1CS; 


render(1 


When a user is logged in, the request object has a request.user attribute 
set, which contains information about the user. The query Topic.objects 
.filter(owner-request.user) tells Django to retrieve only the Topic objects 
from the database whose owner attribute matches the current user. Because 
we're not changing how the topics are displayed, we don't need to change 
the template for the topics page at all. 

To see if this works, log in as the user you connected all existing topics 
to, and go to the topics page. You should see all the topics. Now log out and 
log back in as a different user. You should see the message “No topics have 
been added yet." 


Protecting a User's Topics 


We haven't restricted access to the topic pages yet, so any registered user 
could try a bunch of URLs (like http://locathost:8000/topics/1/) and retrieve 
topic pages that happen to match. 

Try it yourself. While logged in as the user that owns all topics, copy the 
URL or note the ID in the URL of a topic, and then log out and log back in 
as a different user. Enter that topic's URL. You should be able to read the 
entries, even though you're logged in as a different user. 

We'll fix this now by performing a check before retrieving the requested 
entries in the topic() view function: 


learning logs/ from dj: 
views.py Fron 


aut ( 


@ from django. http import Http404 


Ors 1 


T5 m opic.object: 'et(id-topic id) 


it Make sure the topic belongs to the current user. 
e if topic.owner !- request.user: 
raise Http404 
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learning_logs/ 
views. py 


learning_logs/ 
views. py 


A 404 response is a standard error response that’s returned when a 
requested resource doesn’t exist on a server. Here we import the Http404 
exception 8, which we'll raise if the user requests a topic they shouldn't 
have access to. After receiving a topic request, we make sure the topic's 
user matches the currently logged-in user before rendering the page. If the 
requested topic's owner is not the same as the current user, we raise the 
Http404 exception @, and Django returns a 404-error page. 

Now if you try to view another user's topic entries, you'll see a *Page Not 
Found" message from Django. In Chapter 20, we'll configure the project so 
users will see a proper error page instead of a debugging page. 


Protecting the edit entry Page 


The edit entry pages have URLs of the form http://localhost:8000/edit_entry/ 
entry id/, where the entry idis a number. Let's protect this page so no one 
can use the URL to gain access to someone else's entries: 


if topic.owner l= request.user: 
raise Http404 


We retrieve the entry and the topic associated with this entry. We then 
check whether the owner of the topic matches the currently logged-in user; 
if they don't match, we raise an Http404 exception. 


Associating New Topics with the Current User 


Currently, the page for adding new topics is broken because it doesn't 
associate new topics with any particular user. If you try adding a new topic, 
you'll see the message IntegrityError along with NOT NULL constraint failed: 
learning logs topic.owner id. Django is saying you can't create a new topic 
without specifying a value for the topic's owner field. 

There's a straightforward fix for this problem, because we have access 
to the current user through the request object. Add the following code, 
which associates the new topic with the current user: 
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new topic = form.save(commit=False) 
new_topic.owner = request.user 
new_topic.save() 


00o 


When we first call form. save(), we pass the commit=False argument because 
we need to modify the new topic before saving it to the database @. We then 
set the new topic’s owner attribute to the current user @. Finally, we call save() 
on the topic instance we just defined 6. Now the topic has all the required 
data and will save successfully. 

You should be able to add as many new topics as you want for as many 
different users as you want. Each user will only have access to their own data, 
whether they're viewing data, entering new data, or modifying old data. 


TRY IT YOURSELF 


19-3. Refactoring: There are two places in views.py where we make sure the 
user associated with a topic matches the currently logged-in user. Put the code 
for this check in a function called check topic owner(), and call this function 
where appropriate. 


19-4. Protecting new. entry: Currently, a user can add a new entry to another 
user's learning log by entering a URL with the ID of a topic belonging to 
another user. Prevent this attack by checking that the current user owns the 
entry's topic before saving the new entry. 


19-5. Protected Blog: In your Blog project, make sure each blog post is con- 


nected to a particular user. Make sure all posts are publicly accessible but only 
registered users can add posts and edit existing posts. In the view that allows 
users to edit their posts, make sure the user is editing their own post before pro- 
cessing the form. 


Summary 


In this chapter, you learned how forms allow users to add new topics and 
entries, and edit existing entries. You then learned how to implement user 
accounts. You gave existing users the ability to log in and out, and used 
Django’s default UserCreationForm to let people create new accounts. 
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After building a simple user authentication and registration system, you 
restricted access to logged-in users for certain pages using the @login_ required 
decorator. You then assigned data to specific users through a foreign key 
relationship. You also learned to migrate the database when the migration 
requires you to specify some default data. 

Finally, you learned how to make sure a user can only see data that 
belongs to them by modifying the view functions. You retrieved appro- 
priate data using the filter() method, and compared the owner of the 
requested data to the currently logged-in user. 

It might not always be immediately obvious what data you should make 
available and what data you should protect, but this skill will come with 
practice. The decisions we've made in this chapter to secure our users' data 
also illustrate why working with others is a good idea when building a proj- 
ect: having someone else look over your project makes it more likely that 
you'll spot vulnerable areas. 

You now have a fully functioning project running on your local machine. 
In the final chapter, you'll style Learning Log to make it visually appealing, 
and you'll deploy the project to a server so anyone with internet access can 
register and make an account. 
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STYLING AND DEPLOYING 
AN APP 


Learning Log is fully functional now, but 
it has no styling and runs only on your 
local machine. In this chapter, you'll style 

the project in a simple but professional man- 
ner and then deploy it to a live server so anyone in the 
world can make an account and use it. 


For the styling, we'll use the Bootstrap library, a collection of tools for 
styling web applications so they look professional on all modern devices, 
from a small phone to a large desktop monitor. To do this, we'll use the 
django-bootstrap5 app, which will also give you practice using apps made 
by other Django developers. 

We'll deploy Learning Log using Platform.sh, a site that lets you push 
your project to one of its servers, making it available to anyone with an 
internet connection. We'll also start using a version control system called 
Git to track changes to the project. 
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When you're finished with Learning Log, you'll be able to develop 
simple web applications, give them a professional look and feel, and deploy 
them to a live server. You'll also be able to use more advanced learning 
resources as you develop your skills. 


Styling Learning Log 


settings.py 


Chapter 20 


We've purposely ignored styling until now to focus on Learning Log's func- 
tionality first. This is a good way to approach development, because an app 
is only useful if it works. Once an app is working, its appearance is critical 
so people will want to use it. 

In this section, we'll install the django-bootstrap5 app and add it to the 
project. We'll then use it to style the individual pages in the project, so all 
the pages have a consistent look and feel. 


The django-bootstrap5 App 


We'll use django-bootstrap5 to integrate Bootstrap into our project. This 
app downloads the required Bootstrap files, places them in an appropriate 
location in your project, and makes the styling directives available in your 
project's templates. 

To install django-bootstrap5, issue the following command in an active 
virtual environment: 


(1l env)learning log$ pip install django-bootstrap5 

--snip-- 

Successfully installed beautifulsoup4-4.11.1 django-bootstrap5-21.3 
soupsieve-2.3.2.post1 


Next, we need to add django-bootstrap5 to INSTALLED APPS in settings.py: 


# Third party apps. 
'django bootstrap5', 


Start a new section called Third party apps, for apps created by other devel- 
opers, and add 'django bootstraps' to this section. Make sure you place this 
section after My apps but before the section containing Django's default apps. 


Using Bootstrap to Style Learning Log 


Bootstrap is a large collection of styling tools. It also has a number of tem- 
plates you can apply to your project to create an overall style. It's much 
easier to use these templates than to use individual styling tools. To see the 


templates Bootstrap offers, go to https://getbootstrap.com and click Examples. 
We'll use the Navbar static template, which provides a simple top navigation 
bar and a container for the page’s content. 

Figure 20-1 shows what the home page will look like after we apply 
Bootstrap’s template to base.html and modify index.html slightly. 


eee ® iocaihost:8000 


Learning Log r Register Log in 


Track your learning. 


Make your own Learning Log, and keep a list of the topics you're learning 
about. Whenever you learn something new about a topic, make an entry 
summarizing what you've learned. 


Figure 20-1: The Learning Log home page using Bootstrap 


Modifying base.html 


We need to rewrite base.html using the Bootstrap template. We'll develop 
the new base.htmlin sections. This is a large file; you may want to copy this 
file from the online resources, available at hitps://ehmatthes.github.io/pcc_3e. If 
you do copy the file, you should still read through the following section to 
understand the changes that were made. 


Defining the HTML Headers 


The first change we'll make to base.himl defines the HTML headers in the 
file. We'll also add some requirements for using Bootstrap in our templates, 
and give the page a title. Delete everything in base.html and replace it with 
the following code: 


base.html € <!doctype html» 
6 «html lang="en"> 
© <head> 
«meta charset="utf-8"> 
«meta name-"viewport" content="width=device-width, initial-scale=1"> 
© <title>Learning Log</title> 


© {% load django bootstrap5 %} 
{% bootstrap css XX) 
{% bootstrap javascript %} 


</head> 
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We first declare this file as an HTML document 6 written in English 6. 
An HTML file is divided into two main parts: the head and the body. The 
head of the file begins with an opening «head» tag ©. The head of an HTML 
file doesn't hold any of the page's content; it just tells the browser what it 
needs to know to display the page correctly. We include a «title» element for 
the page, which will display in the browser's title bar whenever Learning Log 
is open Q. 

Before closing the head section, we load the collection of template tags 
available in django-bootstrap5 @. The template tag (X bootstrap css %} isa 
custom tag from django-bootstrap5; it loads all of the CSS files required to 
implement Bootstrap styles. The tag that follows enables all the interactive 
behavior you might use on a page, such as collapsible navigation bars. The 
closing </head> tag appears on the last line. 

All Bootstrap styling options are now available in any template that 
inherits from base. html. If you want to use custom template tags from 
django-bootstrap5, each template will need to include the (X load django 
 bootstrap5 %} tag. 


Defining the Navigation Bar 


The code that defines the navigation bar at the top of the page is fairly long, 
because it has to work equally well on narrow phone screens and wide desk- 
top monitors. We'll work through the navigation bar in sections. 

Here's the first part of the navigation bar: 


<body> 
«nav class="navbar navbar-expand-md navbar-light bg-light mb-4 border"» 
<div class="container-fluid"> 


«a class-"navbar-brand" href="{% url ‘learning logs:index' %}"> 
Learning Log</a> 


«button class-"navbar-toggler" type="button" data-bs-toggle-"collapse" 
data-bs-target="#navbarCollapse" aria-controls-"navbarCollapse" 
aria-expanded-"false" aria-label-"Toggle navigation"> 
«span class="navbar-toggler-icon"></span> 

</button> 


<div class-"collapse navbar-collapse" id="navbarCollapse"> 
<ul class-"navbar-nav me-auto mb-2 mb-md-0"> 
«li class-"nav-item"» 
<a class-"nav-link" href="{% url 'learning logs:topics' %}"> 
Topics«/a»«/1li» 
</ul> «!-- End of links on left side of navbar --> 
</div> <!-- Closes collapsible parts of navbar --> 


</div> <!-- Closes navbar's container --> 
</nav> <!-- End of navbar --> 


© {% block content %}{% endblock content %} 


</body> 
</html> 


The first new element is the opening <body> tag. The body of an HTML 
file contains the content users will see on a page. Next we have a <nav> ele- 
ment, which opens the code for the navigation bar at the top of the page 8. 
Everything contained in this element is styled according to the Bootstrap 
style rules defined by the selectors navbar, navbar-expand-md, and the rest that 
you see here. A selector determines which elements on a page a certain style 
rule applies to. The navbar-light and bg-light selectors style the navigation bar 
with a lightthemed background. The mb in mb-4 is short for margin-bottom; this 
selector ensures that a little space appears between the navigation bar and 
the rest of the page. The border selector provides a thin border around the 
light background to set it offa little from the rest of the page. 

The «div» tag on the next line opens a resizable container that will hold 
the overall navigation bar. The term divis short for division; you build a web 
page by dividing it into sections and defining style and behavior rules that 
apply to that section. Any styling or behavior rules that are defined in an 
opening «div» tag affect everything you see until its corresponding closing 
tag, written as «/div». 

Next we set the project's name, Learning Log, to appear as the first ele- 
ment on the navigation bar 6. This will also serve as a link to the home 
page, just as it’s been doing in the minimally styled version of the project we 
built in the previous two chapters. The navbar-brand selector styles this link 
so it stands out from the rest of the links and helps add some branding to 
the site. 

The Bootstrap template then defines a button that appears if the browser 
window is too narrow to display the whole navigation bar horizontally 9. 
When the user clicks the button, the navigation elements appear in a drop- 
down list. The collapse reference causes the navigation bar to collapse when 
the user shrinks the browser window or when the site is displayed on devices 
with small screens. 

Next, we open a new section («div») of the navigation bar O. This is the 
part of the navigation bar that can collapse depending on the size of the 
browser window. 

Bootstrap defines navigation elements as items in an unordered list ©, 
with style rules that make it look nothing like a list. Every link or element 
you need on the bar can be included as an item in an unordered list ©. 
Here, the only item in the list is our link to the topics page 9. Notice the 
closing </li> tag at the end of the link; every opening tag needs a corre- 
sponding closing tag. 

The rest of the lines shown here close out all of the tags that have been 
opened. In HTML, a comment is written like this: 


<!-- This is an HTML comment. --> 
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Closing tags don’t usually have comments, but if you’re new to HTML, 
it can be really helpful to label some of your closing tags. A single missing 
tag or an extra tag can throw off the layout of an entire page. We include 
the content block © and the closing </body> and </html> tags as well. 

We're not finished with the navigation bar, but we now have a complete 
HTML document. If runserver is currently active, stop the current server 
and restart it. Go to the project's home page, and you should see a naviga- 
tion bar that has some of the elements shown in Figure 20-1. Now let's add 
the rest of the elements to the navigation bar. 


Adding User Account Links 


We still need to add the links associated with user accounts. We'll start by 
adding all of the account-related links except the logout form. 
Make the following changes to base.html: 


<!-- Account-related links --> 
«ul class-"navbar-nav ms-auto mb-2 mb-md-0"> 


{% if user.is authenticated %} 
<li class="nav-item"> 
<span class="navbar-text me-2">Hello, {{ user.username }}. 
</span></li> 
{% else %} 
<li class="nav-item"> 
<a class="nav-link" href="{% url 'accounts:register' %}"> 
Register</a></li> 
<li class="nav-item"> 
<a class="nav-link" href="{% url 'accounts:login' %}"> 
Log in</a></li> 
{% endif %} 


</ul> <!-- End of account-related links --> 


We begin a new set of links by using another opening <ul> tag ®. You 
can have as many groups of links as you need on a page. The selector 
ms-auto is short for margin-start-automatic: this selector examines the other 
elements in the navigation bar and works out a left (start) margin that 
pushes this group of links to the right side of the browser window. 

The if block is the same conditional block we used earlier to display 
appropriate messages to users, depending on whether they're logged in @. 
The block is a little longer now because there are some styling rules inside 
the conditional tags. The greeting for authenticated users is wrapped in a 
<span> element 6. A span element styles pieces of text or elements of a page 
that are part of a longer line. While div elements create their own divisions 
in a page, span elements are continuous within a larger section. This can 


base. html 


base. html 


be confusing at first, because many pages have deeply nested div elements. 
Here, we’re using the span element to style informational text on the navi- 
gation bar: in this case, the logged-in user’s name. 

In the else block, which runs for unauthenticated users, we include the 
links for registering a new account and logging in 9. These should look just 
like the link to the topics page. 

If you wanted to add more links to the navigation bar, you'd add another 
«li» item to one of the <ul> groups that we've defined, using styling directives 
like the ones you've seen here. 

Now let's add the logout form to the navigation bar. 


Adding the Logout Form to the Navigation Bar 


When we first wrote the logout form, we added it to the bottom of base.html. 
Now let's put it in a better place, in the navigation bar: 


{% if user.is authenticated %} 
<form action="{% url 'accounts:logout' Xj" method-'post'» 
(^ csrf token %} 
<button name-'submit' class-'btn btn-outline-secondary btn-sm'» 
Log out«/button» 
</form> 
{% endif %} 


The logout form should be placed after the set of account-related links, 
but inside the collapsible section of the navigation bar. The only change in the 
form is the addition of a number of Bootstrap styling classes in the <button> 
element, which apply Bootstrap styling elements to the logout button 6. 

Reload the home page, and you should be able to log in and out using 
any of the accounts you've created. 

There’s still a bit more we need to add to base.himl. We need to define 
two blocks that the individual pages can use to place the content specific to 
those pages. 


Defining the Main Part of the Page 


The rest of base.html contains the main part of the page: 


«main class-"container"» 
«div class-"pb-2 mb-2 border-bottom"» 
(^ block page header %}{% endblock page header %} 
</div> 
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«/div» 
«/main» 


We first open a «main» tag 9. The main element is used for the most 
significant part of the body of a page. Here we assign the bootstrap selector 
container, which is a simple way to group elements on a page. We'll place 
two div elements in this container. 

The first div element contains a page header block @. We'll use this block 
to title most pages. To make this section stand out from the rest of the page, 
we place some padding below the header. Padding refers to space between 
an element’s content and its border. The selector pb-2 is a bootstrap directive 
that provides a moderate amount of padding at the bottom of the styled ele- 
ment. A margin is the space between an element's border and other elements 
on the page. The selector mb-2 provides a moderate amount of margin at the 
bottom of this div. We want a border on the bottom of this block, so we use 
the selector border-bottom, which provides a thin border at the bottom of the 
page header block. 

We then define one more div element that contains the block content 6. 
We don't apply any specific style to this block, so we can style the content of 
any page as we see fit for that page. The end of the base.himl file has closing 
tags for the main, body, and html elements. 

When you load Learning Log's home page in a browser, you should 
see a professional-looking navigation bar that matches the one shown in 
Figure 20-1. Try resizing the window so it's really narrow; a button should 
replace the navigation bar. Click the button, and all the links should appear 
in a drop-down list. 


Styling the Home Page Using a Jumbotron 


To update the home page, we'll use a Bootstrap element called a jumbotron, 
a large box that stands out from the rest of the page. Typically, it’s used on 
home pages to hold a brief description of the overall project and a call to 
action that invites the viewer to get involved. 

Here's the revised index.html file: 


index.html 


6 (4 block page header %} 

@ <div class="p-3 mb-4 bg-light border rounded-3"> 
«div class-"container-fluid py-4"> 

e «hi class-"display-3"»Track your learning.«/hi» 


o «p class-"lead"»Make your own Learning Log, and keep a list of the 


topics you're learning about. Whenever you learn something new 
about a topic, make an entry summarizing what you've learned.</p> 
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e «a class="btn btn-primary btn-lg mt-1" 
href="{% url 'accounts:register' %}">Register &raquo;</a> 
</div> 
</div> 
{% endblock page_header %} 


We first tell Django that we’re about to define what goes in the page 
_header block 6. A jumbotron is implemented as a pair of div elements with 
a set of styling directives applied to them @. The outer div has padding and 
margin settings, a light background color, and rounded corners. The inner 
div is a container that changes along with the window size and has some 
padding as well. The py-4 selector adds padding to the top and bottom of 
the div element. Feel free to adjust the numbers in these settings and see 
how the home page changes. 

Inside the jumbotron are three elements. The first is a short message, 
Track your learning, that gives new visitors a sense of what Learning Log 
does ©. The «h1» element is a first-level header, and the display-3 selector 
adds a thinner and taller look to this particular header. We also include a 
longer message that provides more information about what the user can do 
with their learning log O. This is formatted as a lead paragraph, which is 
meant to stand out from regular paragraphs. 

Rather than just using a text link, we create a button that invites users 
to register an account on Learning Log 9. This is the same link as in the 
header, but the button stands out on the page and shows the viewer what 
they need to do in order to start using the project. The selectors you see 
here style this as a large button that represents a call to action. The code 
&raquo; isan HTML entity that looks like two right angle brackets combined 
(>>). Finally, we provide closing div tags and close the page_header block. 
With only two div elements in this file, it’s not particularly helpful to label 
the closing div tags. We aren't adding anything else to this page, so we don't 
need to define the content block in this template. 

The home page now looks like Figure 20-1. This is a significant improve- 
ment over the unstyled version of the project! 


Styling the Login Page 
We've refined the overall appearance of the login page, but the login form 


itself doesn't have any styling yet. Let's make the form look consistent with 
the rest of the page by modifying login.html: 


login.html ds 'learning logs/base.html 
6 {% load django bootstrap5 %} 
@ {% block page header %} 


<h2>Log in to your account.</h2> 
{% endblock page header %} 
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{% bootstrap form form X) 
(^ bootstrap button button type="submit" content-"Log in" %} 


n» 


We first load the bootstrap5 template tags into this template @. We then 
define the page header block, which tells the user what the page is for 8. 
Notice that we've removed the {% if form.errors %} block from the template; 
django-bootstrap5 manages form errors automatically. 

To display the form, we use the template tag (4 bootstrap form 4) 9; this 
replaces the {{ form.as div }} element we were using in Chapter 19. The 
(X booststrap form 4) template tag inserts Bootstrap style rules into the form’s 
individual elements as the form is rendered. To generate the submit button, 
we use the (4 bootstrap button %} tag with arguments that designate it as a 
submit button, and give it the label Log in O. 

Figure 20-2 shows the login form now. The page is much cleaner, with 
consistent styling and a clear purpose. Try logging in with an incorrect 
username or password; you'll see that even the error messages are styled 
consistently and integrate well with the overall site. 


eee ff 4 € localhost:-8000/eccoumtaflogin’ 
Learning Log Topic Register Login 


Log in to your account. 


Username 


Password 


Figure 20-2: The login page styled with Bootstrap 


Styling the Topics Page 
Let’s make sure the pages for viewing information are styled appropriately 
as well, starting with the topics page: 


4 extends lea ng 10gS/Dase.nur fo f 


(^ block page header Xj 


topic.html 


e 


e 


e 


<h1>Topics</h1> 
{% endblock page header %} 


«ul class-"list-group border-bottom pb-2 mb-4"> 
«li class-"list-group-item border-0"» 
«a href="{% url 'learning logs:topic' topic.id %}"> 
{{ topic.text }}</a> 
</li> 


<li class-"list-group-item border-0">No topics have been added yet.«/li» 


We don't need the {% load bootstrap5 %} tag, because we're not using any 
custom bootstrap5 template tags in this file. We move the heading Topics 
into the page header block and make it an «h1» element instead of a simple 
paragraph 6. 

The main content on this page is a list of topics, so we use Bootstrap's 
list group component to render the page. This applies a simple set of styling 
directives to the overall list and to each item in the list. When we open the 
«ul» tag, we first include the list-group class to apply the default style direc- 
tives to the list @. We further customize the list by putting a border at the 
bottom of the list, a little padding below the list (pb-2), and a margin below 
the bottom border (mb-4). 

Each item in the list needs the list-group-item class, and we customize 
the default style by removing the border around individual items 6. The 
message that's displayed when the list is empty needs these same classes O. 

When you visit the topics page now, you should see a page with styling 
that matches the home page. 


Styling the Entries on the Topic Page 


On the topic page, we'll use Bootstrap's card component to make each 
entry stand out. A card is a nestable set of divs with flexible, predefined 
styles that are perfect for displaying a topic's entries: 


extends  lear 


6 {% block page header %} 


«hi»(( topic.text }}</h1> 
(4 endblock page header %} 
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<div class="card mb-3"> 
<!-- Card header with timestamp and edit link --> 
<h4 class="card-header"> 
{{ entry.date added|date:'M d, Y H:i' }} 
<small><a href="{% url ‘learning logs:edit entry' entry.id %}"> 
edit entry«/a»«/small» 
</h4> 
<!-- Card body with entry text --> 
<div class="card-body">{{ entry.text|linebreaks }}</div> 
</div> 


<p>There are no entries for this topic yet.</p> 


o» endTOI /j 


We first place the topic in the page header block 6. Then we delete the 
unordered list structure previously used in this template. Instead of making 
each entry a list item, we open a div element with the selector card 6. This 
card has two nested elements: one to hold the timestamp and the link to 
edit the entry, and another to hold the body of the entry. The card selector 
takes care of most of the styling we need for this div; we customize the card 
by adding a small margin to the bottom of each card (mb-3). 

The first element in the card is a header, which is an «h4» element with 
the selector card-header ®©. This header contains the date the entry was 
made and a link to edit the entry. The «small» tag around the edit entry link 
makes it appear a little smaller than the timestamp ©. The second element 
is a div with the selector card-body 6, which places the text of the entry in 
asimple box on the card. Notice that the Django code for including the 
information on the page hasn't changed; only elements that affect the 
appearance of the page have. Since we no longer have an unordered list, 
we've replaced the list item tags around the empty list message with simple 
paragraph tags ©. 

Figure 20-3 shows the topic page with its new look. Learning Log's 
functionality hasn't changed, but it looks significantly more professional 
and inviting to users. 

If you want to use a different Bootstrap template for a project, follow a 
process that's similar to what we've done so far in this chapter. Copy the tem- 
plate you want to use into base.himl, and modify the elements that contain 
actual content so the template displays your project's information. Then use 
Bootstrap's individual styling tools to style the content on each page. 


The Bootstrap project has excellent documentation. Visit the home page at https:// 
getbootstrap.com and click Docs to learn more about what Bootstrap offers. 


eee ff < localhost:8000/topics/ " (d + Q 


Learning Log 


Learning Log Topics Hello, Il. admin gout 


Chess 


Add new entry 


May 20, 2022 21:21 edit entry 


The bishops and knights are good pieces to have out in the opening phase of the game. 
They're both powerful enough to be useful in attacking your opponent, but not so powerful 
that you can't afford to lose them in an early trade. 


May 20, 2022 03:44 edit entry 


In the opening phase of the game, it's important to bring out your bishops and knights. These 
pieces are powerful and maneuverable enough to play a significant role in the beginning 
moves of a game. 


May 20, 2022 03:43 edit entry 


The opening is the first part of the game, roughly the first ten moves or so. In the opening, it's 
a good idea to do three things—bring out your bishops and knights, try to control the center 
of the board, and castle your king. 


Of course. these are iust auidelines. It will be important to learn when to follow these 


Figure 20-3: The topic page with Bootstrap styling 


TRY IT YOURSELF 


20-1. Other Forms: We applied Bootstrap's styles to the login page. Make simi- 
lar changes to the rest of the form-based pages, including new topic, new entry, 
edit entry, and register. 


20-2. Stylish Blog: Use Bootstrap to style the Blog project you created in 
Chapter 19. 


Deploying Learning Log 


Now that we have a professional-looking project, let's deploy it to a live 
server so anyone with an internet connection can use it. We'll use Platform.sh, 
a web-based platform that allows you to manage the deployment of web 
applications. We'll get Learning Log up and running on Platform.sh. 


Making a Platform.sh Account 


To make an account, go to Attps://platform.sh and click the Free Trial button. 
Platform.sh has a free tier that, as of this writing, does not require a credit 
card. The trial period allows you to deploy an app with minimal resources, 
which lets you test your project in a live deployment before committing to a 
paid hosting plan. 
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The specific limits of trial plans tend to change periodically, as hosting platforms 
fight spam and abuse of resources. You can see the current limits of the free trial at 
https://platform.sh/free-trial. 


Installing the Platform.sh CLI 


To deploy and manage a project on Platform.sh, you'll need the tools avail- 
able in the Command Line Interface (CLI). To install the latest version 
of the CLI, visit Attps://docs.platform.sh/development/cli. html and follow the 
instructions for your operating system. 

On most systems, you can install the CLI by running the following com- 
mand in a terminal: 


$ curl -fsS https://platform.sh/cli/installer | php 


After this command has finished running, you will need to open a new 
terminal window before you can use the CLI. 


This command will probably not work in a standard terminal on Windows. You 
can use Windows Subsystem for Linux (WSL) or a Git Bash terminal. If you need to 
install PHP, you can use the XAMPP installer from https://apachefriends.org. If 
you have any difficulty installing the Platform.sh CLI, see the more detailed installa- 
lion instructions in Appendix E. 


Installing platformshconfig 


You'll also need to install one additional package, platformshconfig. T'his 
package helps detect whether the project is running on your local system or 
on a Platform.sh server. In an active virtual environment, issue the follow- 
ing command: 


(1l env)learning log$ pip install platformshconfig 


We'll use this package to modify the project's settings when it's running 
on the live server. 


Creating a requirements.txt File 


The remote server needs to know which packages Learning Log depends 
on, so we'll use pip to generate a file listing them. Again, from an active vir- 
tual environment, issue the following command: 


(1l env)learning log$ pip freeze > requirements.txt 


The freeze command tells pip to write the names of all the packages 
currently installed in the project into the file requirements.txt. Open this file 
to see the packages and version numbers installed in your project: 


asgiref--3.5.2 
beautifulsoup4==4.11.1 
Django==4.1 


requirements 
_remote. txt 


django-bootstrap5==21.3 
platformshconfig==2.4.0 
soupsieve==2.3.2.post1 
sqlparse==0.4.2 


Learning Log already depends on specific versions of seven differ- 
ent packages, so it requires a matching environment to run properly on a 
remote server. (We installed three of these packages manually, and four of 
them were installed automatically as dependencies of these packages.) 

When we deploy Learning Log, Platform.sh will install all the packages 
listed in requirements.txt, creating an environment with the same packages 
we're using locally. Because of this, we can be confident the deployed project 
will function just like it has on our local system. This approach to managing 
a project is critical as you start to build and maintain multiple projects on 
your system. 


If the version number for a package listed on your system differs from what's shown 
here, keep the version you have on your system. 


Additional Deployment Requirements 


The live server requires two additional packages. These packages are used 
to serve the project in a production environment, where many users can be 
making requests at the same time. 

In the same directory where requirements.txt is saved, make a new file 
called requirements remote.txt. Add the following two packages to it: 


# Requirements for live project. 
gunicorn 


psycopg2 


The gunicorn package responds to requests as they come in to the 
remote server; this takes the place of the development server we’ve been 
using locally. The psycopg2 package is required to let Django manage the 
Postgres database that Platform.sh uses. Postgres is an open source database 
that’s extremely well suited to production apps. 


Adding Configuration Files 


Every hosting platform requires some configuration for a project to run 
correctly on its servers. In this section, we’ll add three configuration files: 


-platform.app.yaml This is the main configuration file for the project. 
This tells Platform.sh what kind of project we're trying to deploy and 
what kinds of resources our project needs, and it includes commands 
for building the project on the server. 


platform/routes.yaml This file defines the routes to our project. When 
a request is received by Platform.sh, this is the configuration that helps 
direct these requests to our specific project. 

.platform/services.yaml This file defines any additional services our 
project needs. 
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These are all YAML (YAML Ain't Markup Language) files. YAML is 
a language designed for writing configuration files; it’s made to be read 
easily by both humans and computers. You can write or modify a typical 
YAML file by hand, but a computer can also read and interpret the file 
unambiguously. 

YAML files are great for deployment configuration, because they give 
you a good deal of control over what happens during the deployment 
process. 


Making Hidden Files Visible 


Most operating systems hide files and folders that begin with a dot, such as 
.platform. When you open a file browser, you won't see these kinds of files 
and folders by default. But as a programmer, you'll need to see them. Here's 
how to view hidden files, depending on your operating system: 


e On Windows, open Windows Explorer, and then open a folder such as 
Desktop. Click the View tab, and make sure File name extensions and 
Hidden items are checked. 


e On macOS, you can press 38-SHIFT-. (dot) in any Finder window to see 
hidden files and folders. 


e On Linux systems such as Ubuntu, you can press CTRL-H in any file 
browser to display hidden files and folders. To make this setting per- 
manent, open a file browser such as Nautilus and click the options tab 
(indicated by three lines). Select the Show Hidden Files checkbox. 


The .platform.app.yaml Configuration File 


The first configuration file is the longest, because it controls the overall 
deployment process. We'll show it in parts; you can either enter it by hand 
in your text editor or download a copy from the online resources at hitps:// 
ehmatthes.github.io/bcc 3e. 

Here's the first part of .platform.app.yaml, which should be saved in the 
same directory as manage.py: 


„platform € name: "1l project" 
.app.yaml type: "python:3.10" 


@ relationships: 
database: "db:postgresql" 


# The configuration of the app when it's exposed to the web. 

© web: 
upstream: 

socket_family: unix 
commands : 

start: "gunicorn -w 4 -b unix:$SOCKET 11 project.wsgi:application" 
locations: 

"ns 

passthru: true 
"/static": 


ao 
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(6) 


e 


e 


e 


o 


e 


root: "static" 
expires: 1h 
allow: true 


# The size of the persistent disk of the application (in MB). 
disk: 512 


When you save this file, make sure you include the dot at the beginning 
of the filename. If you omit the dot, Platform.sh won't find the file and your 
project will not be deployed. 

You don't need to understand everything in .platform.app.yaml at this 
point; lll highlight the most important parts of the configuration. The file 
starts off by specifying the name of the project, which we're calling '11 project' 
to be consistent with the name we used when starting the project 0. We also 
need to specify the version of Python we're using (3.10 at the time of this 
writing). You can find a list of supported versions at hitps://docs.platform.sh/ 
languages/python.himl. 

Next is a section labeled relationships that defines other services the proj- 
ect needs 9. Here the only relationship is to a Postgres database. After that 
is the web section ©. The commands: start section tells Platform.sh what process 
to use to serve incoming requests. Here we're specifying that gunicorn will 
handle requests O. This command takes the place of the python manage. py 
runserver command we've been using locally. 

The locations section tells Platform.sh where to send incoming requests 9. 
Most requests should be passed through to gunicorn; our urls.py files will tell 
gunicorn exactly how to handle those requests. Requests for static files will be 
handled separately and will be refreshed once an hour. The last line shows 
that we're requesting 512MB of disk space on one of Platform.sh’s servers ©. 

The rest of .platform.app.yamlis as follows: 


# Set a local read/write mount for logs. 
mounts: 
"logs": 
source: local 
source path: logs 


# The hooks executed at various points in the lifecycle of the application. 
hooks: 
build: | 
pip install --upgrade pip 
pip install -r requirements. txt 
pip install -r requirements remote.txt 


mkdir logs 
python manage.py collectstatic 
rm -rf logs 
deploy: | 
python manage.py migrate 
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E platform/ 
routes.yaml 


_platform/ 
routes.yaml 
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The mounts section 6 lets us define directories where we can read and 
write data while the project is running. This section defines a logs/ directory 
for the deployed project. 

The hooks section @ defines actions that are taken at various points 
during the deployment process. In the build section, we install all the pack- 
ages that are required to serve the project in the live environment ©. We 
also run collectstatic @, which collects all the static files needed for the 
project into one place so they can be served efficiently. 

Finally, in the deploy section 6, we specify that migrations should be 
run each time the project is deployed. In a simple project, this will have no 
effect when there have been no changes. 

The other two configuration files are much shorter; let’s write them now. 


The routes.yaml Configuration File 


A route is the path a request takes as it’s processed by the server. When 
a request is received by Platform.sh, it needs to know where to send the 
request. 

Make a new folder called .platform, in the same directory as manage.py. 
Make sure you include the dot at the beginning of the name. Inside that 
folder, make a file called routes.yaml and enter the following: 


# Each route describes how an incoming URL will be processed by Platform.sh. 


"https://{default}/": 
type: upstream 
upstream: "ll project:http" 


"https://www. {default}/": 
type: redirect 
to: "https: //(default]/" 


This file makes sure requests like Attps;//project. url.com and www.project 
_url.com all get routed to the same place. 


The services.yaml Configuration File 


This last configuration file specifies services that our project needs in order 
to run. Save this file in the .platform/ directory, alongside routes.yaml: 


# Each service listed will be deployed in its own container as part of your 
# — Platform.sh project. 


db: 


type: postgresql:12 
disk: 1024 


This file defines one service, a Postgres database. 


settings.py 


e 


e 


e 


o 


e 


Modifying settings.py for Platform.sh 


Now we need to add a section at the end of settings.py to modify some settings 
for the Platform.sh environment. Add this code to the very end of settings.py: 


# Platform.sh settings. 
from platformshconfig import Config 


config - Config() 
if config.is valid platform(): 
ALLOWED HOSTS.append('.platformsh.site') 


if config.appDir: 

STATIC ROOT - Path(config.appDir) / 'static' 
if config.projectEntropy: 

SECRET KEY = config.projectEntropy 


if not config.in build(): 
db settings - config.credentials('database') 
DATABASES - ( 
'default': { 
'ENGINE': 'django.db.backends.postgresql', 
'NAME': db settings['path'], 
'USER': db settings['username'], 
"PASSWORD': db settings['password'], 
'HOST': db settings['host'], 
'PORT': db settings['port'], 
b 
} 


We normally place import statements at the beginning of a module, but 
in this case, it’s helpful to keep all the remote-specific settings in one sec- 
tion. Here we import Config from platformshconfig 6, which helps determine 
settings on the remote server. We only modify settings if the method config 
.is valid platform() returns True , indicating the settings are being used 
on a Platform.sh server. 

We modify ALLOWED HOSTS to allow the project to be served by hosts end- 
ing in .platformsh.site ©. All projects deployed to the free tier will be served 
using this host. If settings are being loaded in the deployed app's direc- 
tory O, we set STATIC ROOT so that static files are served correctly. We also set 
a more secure SECRET KEY on the remote server 8. 

Finally, we configure the production database @. This is only set if the 
build process has finished running and the project is being served. Everything 
you see here is necessary to let Django talk to the Postgres server that 
Platform.sh set up for the project. 


Using Git to Track the Project’s Files 


As discussed in Chapter 17, Git is a version control program that allows you 
to take a snapshot of the code in your project each time you implement a 
new feature successfully. If anything goes wrong, you can easily return to 
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the last working snapshot of your project; for example, if you accidentally 
introduce a bug while working on a new feature. Each snapshot is called a 
commit. 

Using Git, you can try implementing new features without worrying 
about breaking your project. When you're deploying to a live server, you 
need to make sure you're deploying a working version of your project. To 
read more about Git and version control, see Appendix D. 


Installing Git 


Git may already be installed on your system. To find out, open a new termi- 
nal window and issue the command git --version: 


(11_env)learning log$ git --version 
git version 2.30.1 (Apple Git-130) 


If you get a message indicating that Git is not installed, see the installa- 
tion instructions in Appendix D. 


Configuring Git 

Git keeps track of who makes changes to a project, even when only one per- 
son is working on the project. To do this, Git needs to know your username 
and email. You must provide a username, but you can make up an email for 
your practice projects: 


(11_env)learning log$ git config --global user.name "eric" 
(11_env)learning log$ git config --global user.email "eric@example.com" 


If you forget this step, Git will prompt you for this information when 
you make your first commit. 


Ignoring Files 

We don't need Git to track every file in the project, so we'll tell it to ignore 
some files. Create a file called .gitignorein the folder that contains manage.py. 
Notice that this filename begins with a dot and has no file extension. Here's 
the code that goes in .gitignore: 


ll env/ 
. pycache / 
*.sglite3 


We tell Git to ignore the entire /]. env directory, because we can re-create 
it automatically at any time. We also don't track the __pycache__ directory, 
which contains the .pyc files that are created automatically when the .py files 
are executed. We don't track changes to the local database, because it's a 
bad habit: if you're ever using SOLite on a server, you might accidentally 
overwrite the live database with your local test database when you push the 
project to the server. The asterisk in *.sqlite3 tells Git to ignore any file that 
ends with the extension .sqlite3. 


If yow're using macOS, add .DS Store to your .gitignore file. This is a file that 
stores information about folder settings on macOS, and it has nothing to do with this 
project. 


Committing the Project 


We need to initialize a Git repository for Learning Log, add all the nec- 
essary files to the repository, and commit the initial state of the project. 
Here's how to do that: 


© (11 env)learning log$ git init 
Initialized empty Git repository in /Users/eric/.../learning log/.git/ 
@ (11 env)learning log$ git add . 
© (11 env)learning log$ git commit -am "Ready for deployment to Platform.sh." 
[main (root-commit) c7ffaad] Ready for deployment to Platform.sh. 
42 files changed, 879 insertions(+) 
create mode 100644 .gitignore 
create mode 100644 .platform.app.yaml 
--snip-- 
create mode 100644 requirements remote.txt 
O (11 env)learning log$ git status 
On branch main 
nothing to commit, working tree clean 
(1l env)learning log$ 


We issue the git init command to initialize an empty repository in the 
directory containing Learning Log 6. We then use the git add . command, 
which adds all the files that aren't being ignored to the repository 9. (Don't 
forget the dot.) Next, we issue the command git commit -am "commit message": 
the -a flag tells Git to include all changed files in this commit, and the -m 
flag tells Git to record a log message 8. 

Issuing the git status command @ indicates that we're on the main 
branch and that our working tree is clean. This is the status you'll want to 
see anytime you push your project to a remote server. 


Creating a Project on Platform.sh 


At this point, the Learning Log project still runs on our local system and is 
also configured to run correctly on a remote server. We'll use the Platform.sh 
CLI to create a new project on the server and then push our project to the 
remote server. 

Make sure you're in a terminal, at the learning. log/ directory, and issue 
the following command: 


(1l env)learning log$ platform login 
Opened URL: http://127.0.0.1:5000 
Please use the browser to log in. 
--snip-- 
6 Do you want to create an SSH configuration file automatically? [Y/n] Y 
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This command will open a browser tab where you can log in. Once 
you're logged in, you can close the browser tab and return to the terminal. 
If you're prompted about creating an SSH configuration file 8, enter Y so 
you can connect to the remote server later. 

Now we'll create a project. There's a lot of output, so we'll look at the 
creation process in sections. Start by issuing the create command: 


(1l env)learning log$ platform create 
* Project title (--title) 
Default: Untitled Project 

9 > 11 project 


* Region (--region) 
The region where the project will be hosted 
--snip-- 
[us-3.platform.sh] Moses Lake, United States (AZURE) [514 gCO2eq/kWh] 
@ > us-3.platform.sh 
* Plan (--plan) 
Default: development 
Enter a number to choose: 
[0] development 
--snip-- 
e»o 


* Environments (--environments) 
The number of environments 
Default: 3 

@>3 


* Storage (--storage) 
The amount of storage per environment, in GiB 
Default: 5 

@®>5 


The first prompt asks for a name for the project 6, so we use the name 
1l project. The next prompt asks which region we'd like the server to be 
in @. Choose the server closest to you; for me, that’s us-3.platform.sh. For 
the rest of the prompts, you can accept the defaults: a server on the lowest 
development plan 6, three environments for the project O, and 5GB of 
storage for the overall project 8. 

There are three more prompts to respond to: 


Default branch (--default-branch) 
The default Git branch name for the project (the production environment) 
Default: main 

@ > main 


Git repository detected: /Users/eric/.../learning log 
@ Set the new project 11 project as the remote for this repository? [Y/n] Y 


The estimated monthly cost of this project is: $10 USD 
© Are you sure you want to continue? [Y/n] Y 


454 Chapter 20 


The Platform.sh Bot is activating your project 


The project is now ready! 


A Git repository can have multiple branches; Platform.sh is asking us if 
the default branch for the project should be main @. It then asks if we want 
to connect the local project's repository to the remote repository 6. Finally, 
we're informed that this project will cost about $10 per month if we keep 
it running beyond the free trial period 6. If you haven't entered a credit 
card yet, you shouldn't have to worry about this cost. Platform.sh will simply 
suspend your project if you exceed the free trial's limits without adding a 
credit card. 


Pushing to Platform.sh 


The last step before seeing the live version of the project is to push our code 
to the remote server. To do that, issue the following command: 


(1l env)learning log$ platform push 


6 Are you sure you want to push to the main (production) branch? [Y/n] Y 


--snip-- 
The authenticity of host 'git.us-3.platform.sh (...)' can't be established. 
RSA key fingerprint is SHA256:Tvn...7PM 


O Are you sure you want to continue connecting (yes/no/[fingerprint])? Y 


Pushing HEAD to the existing environment main 


--snip-- 
To git.us-3.platform.sh:3pp3mqcexhlvy.git 
* [new branch] HEAD -> main 


When you issue the command platform push, you'll be asked for one 
more confirmation that you want to push the project ®. You may also see a 
message about the authenticity of Platform.sh, if this is your first time con- 
necting to the site @. Enter Y for each of these prompts, and you'll see a 
bunch of output scroll by. This output will probably look confusing at first, 
but if anything goes wrong, it's really useful to have during troubleshooting. 
If you skim through the output, you can see where Platform.sh installs nec- 
essary packages, collects static files, applies migrations, and sets up URLs 
for the project. 


You may see an error from something that you can easily diagnose, such as a typo 
in one of the configuration files. If this happens, fix the error in your text editor, 
save Lhe file, and reissue the git comnit command. Then you can run platform 
push again. 


Styling and Deploying an App 455 


456 


Chapter 20 


Viewing the Live Project 


Once the push is complete, you can open the project: 


(11_env)learning log$ platform url 

Enter a number to open a URL 
[0] https: //main-bvxea6i-wmye2fx7wwqgu.us-3.platformsh.site/ 
--snip-- 

>0 


The platform url command lists the URLs associated with a deployed 
project; you'll be given a choice of several URLs that are all valid for your 
project. Choose one, and your project should open in a new browser tab! 
This will look just like the project we’ve been running locally, but you can 
share this URL with anyone in the world, and they can access and use your 
project. 


When you deploy your project using a trial account, don't be surprised if it sometimes 
takes longer than usual for a page to load. On most hosting platforms, free resources 
that are idle are often suspended and only restarted when new requests come in. Most 
platforms are much more responsive on paid hosting plans. 


Refining the Platform.sh Deployment 


Now we'll refine the deployment by creating a superuser, just as we did locally. 
We'll also make the project more secure by changing the setting DEBUG to False, 
so error messages won’t show users any extra information that they could use 
to attack the server. 


Creating a Superuser on Platform.sh 


The database for the live project has been set up, but it’s completely empty. 
All the users we created earlier only exist in our local version of the project. 

To create a superuser on the live version of the project, we’ll start an 
SSH (secure socket shell) session where we can run management com- 
mands on the remote server: 


(11_env)learning log$ platform environment: ssh 


Welcome to Platform.sh. 


@ web@ll_project.o:~$ ls 
accounts learning logs 1l project logs manage.py requirements.txt 
requirements remote.txt static 
© webàll project.0:^$ python manage.py createsuperuser 
© Username (leave blank to use 'web'): ll admin live 
Email address: 


Password: 
Password (again): 
Superuser created successfully. 


© webüll project.0:^$ exit 


logout 
Connection to ssh.us-3.platform.sh closed. 


6 (11 env)learning log$ 


When you first run the platform environment:ssh command, you may get 
another prompt about the authenticity of this host. If you see this message, 
enter Y and you should be logged in to a remote terminal session. 

After running the ssh command, your terminal acts just like a terminal on 
the remote server. Note that your prompt has changed to indicate that you're 
in a web session associated with the project named 11 project ®. If you issue 
the 1s command, you'll see the files that have been pushed to the Platform.sh 
server. 

Issue the same createsuperuser command we used in Chapter 18 @. This 
time, I entered an admin username, 11 admin live, that’s distinct from the 
one I used locally ©. When you're finished working in the remote terminal 
session, enter the exit command O. Your prompt will indicate that you're 
working in your local system again 6. 

Now you can add /admin/ to the end of the URL for the live app and 
log in to the admin site. If others have already started using your project, be 
aware that you'll have access to all their data! Take this responsibility seri- 
ously, and users will continue to trust you with their data. 


Windows users will use the same commands shown here (such as 1s instead of dir), 
because you're running a Linux terminal through a remote connection. 


Securing the Live Project 


There's one glaring security issue in the way our project is currently deployed: 
the setting DEBUG = True in settings.py, which provides debug messages when 
errors occur. Django's error pages give you vital debugging information when 
you're developing a project; however, they give way too much information to 
attackers if you leave them enabled on a live server. 

To see how bad this is, go to the home page of your deployed project. 
Log in to a user's account and add /topics/999/ to the end of the home page 
URL. Assuming you haven't made thousands of topics, you should see a 
page with the message DoesNotExist at /topics/999/. If you scroll down, you 
should see a whole bunch of information about the project and the server. 
You won't want your users to see this, and you certainly wouldn't want this 
information available to anyone interested in attacking the site. 

We can prevent this information from being shown on the live site 
by setting DEBUG = False in the part of settings.py that only applies to the 
deployed version of the project. This way you'll continue to see debugging 
information locally, where that information is useful, but it won't show up 
on the live site. 
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Open settings.py in your text editor, and add one line of code to the part 
that modifies settings for Platform.sh: 


DEBUG - False 


All the work to set up configuration for the deployed version of the 
project has paid off. When we want to adjust the live version of the project, 
we just change the relevant part of the configuration we set up earlier. 


Committing and Pushing Changes 


Now we need to commit the changes made to settings.py and push the 
changes to Platform.sh. Here's a terminal session showing the first part of 
this process: 


@ (11 env)learning log$ git commit -am "Set DEBUG False on live site." 


[main d2adof7] Set DEBUG False on live site. 
1 file changed, 1 insertion(+) 


© (11 env)learning log$ git status 


On branch main 
nothing to commit, working tree clean 
(1l env)learning log$ 


We issue the git commit command with a short but descriptive commit 
message 60. Remember the -am flag makes sure Git commits all the files that 
have changed and records the log message. Git recognizes that one file has 
changed and commits this change to the repository. 

Running git status shows that we're working on the main branch of the 
repository and that there are now no new changes to commit 6. It's impor- 
tant to check the status before pushing to a remote server. If you don't see a 
clean status, then some changes haven't been committed and those changes 
won't be pushed to the server. You can try issuing the commit command 
again; if you're not sure how to resolve the issue, read through Appendix D 
to better understand how to work with Git. 

Now let's push the updated repository to Platform.sh: 


(1l env)learning log$ platform push 
Are you sure you want to push to the main (production) branch? [Y/n] Y 
Pushing HEAD to the existing environment main 
--snip-- 
To git.us-3.platform.sh:wmye2fx7wwqgu.git 
fce0206..d2ad0f7 HEAD -> main 
(1l env)learning log$ 


Platform.sh recognizes that the repository has been updated, and it 
rebuilds the project to make sure all the changes have been taken into 
account. It doesn't rebuild the database, so we haven't lost any data. 


404. htm! 


500.html 


settings.py 


To make sure this change took effect, visit the /topics/999/ URL again. 
You should see just the message Server Error (500), with no sensitive informa- 
tion about the project at all. 


Creating Custom Error Pages 


In Chapter 19, we configured Learning Log to return a 404 error if the user 
requests a topic or entry that doesn't belong to them. Now you've seen 

a 500 server error as well. A 404 error usually means your Django code 
is correct, but the object being requested doesn't exist. A 500 error usu- 
ally means there's an error in the code you've written, such as an error in 
a function in views.py. Django currently returns the same generic error 
page in both situations, but we can write our own 404 and 500 error page 
templates that match Learning Log's overall appearance. These templates 
belong in the root template directory. 


Making Custom Templates 


In the learning_log folder, make a new folder called templates. Then make 
a new file called 404.html; the path to this file should be learning. log/ 
templates/404. html. Here's the code for this file: 


{% extends "learning logs/base.html" X) 


(^ block page header %} 
«h2»The item you requested is not available. (404)«/h2» 
{% endblock page header %} 


This simple template provides the generic 404 error page information 
but is styled to match the rest of the site. 
Make another file called 500.himl using the following code: 


{% extends "learning logs/base.html" X) 


(^ block page header %} 
«h2»There has been an internal error. (500)«/h2» 
{% endblock page header %} 


These new files require a slight change to settings.py. 


'DIRS': [BASE DIR / 'templates'], 


Styling and Deploying an App 459 


460 


Chapter 20 


This change tells Django to look in the root template directory for the 
error page templates and any other templates that aren’t associated with a 
particular app. 


Pushing the Changes to Platform.sh 


Now we need to commit the changes we just made and push them to 
Platform.sh: 


© (11 env)learning log$ git add . 
© (11 env)learning log$ git commit -am "Added custom 404 and 500 error pages." 
3 files changed, 11 insertions(*), 1 deletion(-) 
create mode 100644 templates/404.html 
create mode 100644 templates/500.html 
© (11 env)learning log$ platform push 
--snip-- 
To git.us-3.platform.sh:wmye2fx7wwqgu.git 
d2adOf7..9fO042ef HEAD -> main 
(1l env)learning log$ 


We issue the git add . command 6 because we created some new files 
in the project. Then we commit the changes @ and push the updated proj- 
ect to Platform.sh 6. 

Now when an error page appears, it should have the same styling as the 
rest of the site, making for a smoother user experience when errors arise. 


Ongoing Development 


You might want to further develop Learning Log after your initial push to a 
live server, or you might want to develop your own projects to deploy. When 
doing so, there's a fairly consistent process for updating your projects. 

First, you'll make the necessary changes to your local project. If your 
changes result in any new files, add those files to the Git repository using 
the command git add . (making sure to include the dot at the end of the 
command). Any change that requires a database migration will need this 
command, because each migration generates a new migration file. 

Second, commit the changes to your repository using git commit -am 
"commit message". Then push your changes to Platform.sh, using the com- 
mand platform push. Visit your live project and make sure the changes you 
expect to see have taken effect. 

It's easy to make mistakes during this process, so don't be surprised 
when something goes wrong. If the code doesn't work, review what you've 
done and try to spot the mistake. If you can't find the mistake or you 
can't figure out how to undo it, refer to the suggestions for getting help in 
Appendix C. Don't be shy about asking for help: everyone else learned to build 
projects by asking the same questions you're likely to ask, so someone will be 
happy to help you. Solving each problem that arises helps you steadily develop 
your skills until you're building meaningful, reliable projects and answering 
other people's questions as well. 


Deleting a Project on Platform.sh 


It’s great practice to run through the deployment process a number of 

times with the same project or with a series of small projects, to get the 

hang of deployment. But you’ll need to know how to delete a project that’s 

been deployed. Platform.sh also limits the number of projects you can host 

for free, and you don’t want to clutter your account with practice projects. 
You can delete a project using the CLI: 


(11_env)learning log$ platform project:delete 


You'll be asked to confirm that you want to take this destructive action. 
Respond to the prompts, and your project will be deleted. 

The command platform create also gave the local Git repository a refer- 
ence to the remote repository on Platform.sh's servers. You can remove this 
remote from the command line as well: 


(1l env)learning log$ git remote 
platform 
(1l env)learning log$ git remote remove platform 


The command git remote lists the names of all remote URLs associated 
with the current repository. The command git remote remove remote name 
deletes these remote URLs from the local repository. 

You can also delete a project's resources by logging in to the Platform.sh 
website and visiting your dashboard at hitps://console.platform.sh. This page 
lists all your active projects. Click the three dots in a project's box, and click 
Edit Plan. This is a pricing page for the project; click the Delete Project 
button at the bottom of the page, and you'll be shown a confirmation page 
where you can follow through with the deletion. Even if you deleted your 
project using the CLI, it's a good idea to familiarize yourself with the dash- 
board of any hosting provider you deploy to. 


Deleting a project on Platform.sh does nothing to your local version of the project. 
If no one has used your deployed project and you're just practicing the deployment 
process, it’s perfectly reasonable to delete your project on Platform.sh and redeploy it. 
Just be aware that if things stop working, you may have run into the host’s free-tier 

limitations. 


TRY IT YOURSELF 


20-3. Live Blog: Deploy the Blog project you've been working on to Platform.sh. 
Make sure you set DEBUG to False, so users don't see the full Django error pages 


when something goes wrong. 
(continued) 
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20-4. Extended Learning Log: Add one feature to Learning Log, and push the 
change to your live deployment. Try a simple change, such as writing more 
about the project on the home page. Then try adding a more advanced feature, 
such as giving users the option of making a topic public. This would require an 
attribute called public as part of the Topic model (this should be set to False 

by default) and a form element on the new_topic page that allows the user to 


change a topic from private to public. You'd then need to migrate the project 


and revise views.py so any topic that's public is visible to unauthenticated users 
as well. 


Summary 


Chapter 20 


In this chapter, you learned to give your projects a simple but professional 
appearance using the Bootstrap library and the django-bootstrap5 app. 

With Bootstrap, the styles you choose will work consistently on almost any 
device people use to access your project. 

You learned about Bootstrap's templates and used the Navbar static tem- 
plate to create a simple look and feel for Learning Log. You used a jumbo- 
tron to make a home page's message stand out, and learned to style all the 
pages in a site consistently. 

In the final part of the project, you learned how to deploy a project to 
a remote server so anyone can access it. You made a Platform.sh account 
and installed some tools that help manage the deployment process. You 
used Git to commit the working project to a repository, and then pushed 
the repository to a remote server on Platform.sh. Finally, you learned to 
begin securing your app by setting DEBUG - False on the live server. You also 
made custom error pages, so the inevitable errors that come up will look 
well-handled. 

Now that you've finished Learning Log, you can start building your own 
projects. Start simple, and make sure the project works before adding com- 
plexity. Enjoy your continued learning, and good luck with your projects! 


INSTALLATION AND 
TROUBLESHOOTING 


There are many versions of Python avail- 
able and numerous ways to set it up on 


each operating system. If the approach in 
Chapter 1 didn’t work, or if you want to install 

a different version of Python than the one currently 

installed, the instructions in this appendix can help. 


Python on Windows 


The instructions in Chapter 1 show you how to install Python using the 
official installer at https://python.org. If you couldn't get Python to run after 
using the installer, the troubleshooting instructions in this section should 
help you get Python up and running. 


Using py Instead of python 


If you run a recent Python installer and then issue the command python in 
a terminal, you should see the Python prompt for a terminal session (>>>). 
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When Windows doesn’t recognize the python command, it will either open 
the Microsoft Store because it thinks Python isn’t installed, or you'll get a 
message such as “Python was not found.” If the Microsoft Store opens, close 
it; it’s better to use the official Python installer from https://python.org than 
the one that Microsoft maintains. 

The simplest solution, without making any changes to your system, is to 
try the py command. This is a Windows utility that finds the latest version of 
Python installed on your system and runs that interpreter. If this command 
works and you want to use it, simply use py anywhere you see the python or 
python3 command in this book. 


Rerunning the Installer 


The most common reason python doesn’t work is that people forget to select 
the Add Python to PATH option when running the installer; this is an 
easy mistake to make. The PATH variable is a system setting that tells Python 
where to look for commonly used programs. In this case, Windows doesn’t 
know how to find the Python interpreter. 

The simplest fix in this situation is to run the installer again. If there’s a 
newer installer available from hitps://python.org, download the new installer 
and run it, making sure to check the Add Python to PATH box. 

If you already have the latest installer, run it again and select the Modify 
option. You'll see a list of optional features; keep the default options selected 
on this screen. Then click Next and check the Add Python to Environment 
Variables box. Finally, click Install. The installer will recognize that Python is 
already installed, and it will add the location of the Python interpreter to the 
PATH variable. Make sure you close any open terminals, because they'll still be 
using the old PATH variable. Open a new terminal window and issue the com- 
mand python again; you should see a Python prompt (>>>). 


Python on macOS 
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The installation instructions in Chapter 1 use the official Python installer 
at Attps://bython.org. The official installer has been working well for years 
now, but there are a few things that can get you off track. This section will 
help if anything isn't working in a straightforward manner. 


Accidentally Installing Apple's Version of Python 


If you run the python3 command and Python is not yet installed on your 
system, you'll most likely see a message that the command line developer tools 
need to be installed. The best approach at this point is to close the pop-up 
showing this message, download the Python installer from https://python. org, 
and run the installer. 

If you choose to install the command line developer tools at this point, 
macOS will install Apple's version of Python along with the developer tools. 
The only issue with this is that Apple's version of Python is usually some- 
what behind the latest official version of Python. However, you can still 
download and run the official installer from hitps://python.org, and python3 


will then point to the newer version. Don’t worry about having the devel- 
oper tools installed; there are some useful tools in there, including the Git 
version control system discussed in Appendix D. 


Python 2 on Older Versions of macOS 


On older versions of macOS, before Monterey (macOS 12), an outdated ver- 
sion of Python 2 was installed by default. On these systems, the command 
python points to the outdated system interpreter. If you’re using a version of 
macOS with Python 2 installed, make sure you use the python3 command, 
and you'll always be using the version of Python you installed. 


Python on Linux 


Python is included by default on almost every Linux system. However, if the 
default version on your system is earlier than Python 3.9, you should install 
the latest version. You can also install the latest version if you want the most 
recent features, like Python's improved error messages. The following instruc- 
tions should work for most apt-based systems. 


Using the Default Python Installation 


If you want to use the version of Python that python3 points to, make sure 
you have these three additional packages installed: 


$ sudo apt install python3-dev python3-pip python3-venv 


These packages include tools that are useful for developers and tools 
that let you install third-party packages, like the ones used in the projects 
section of this book. 


Installing the Latest Version of Python 


We'll use a package called deadsnakes, which makes it easy to install multiple 
versions of Python. Enter the following commands: 


$ sudo add-apt-repository ppa:deadsnakes/ppa 
$ sudo apt update 
$ sudo apt install python3.11 


These commands will install Python 3.11 onto your system. 
Enter the following command to start a terminal session that runs 
Python 3.11: 


$ python3.11 
>>> 


Anywhere you see the command python in this book, use python3.11 
instead. You'll also want to use this command when you run programs 
from the terminal. 


Installation and Troubleshooting 465 


466 


You'll need to install two more packages to make the most of your Python 
installation: 


$ sudo apt install python3.11-dev python3.11-venv 


These packages include modules you'll need when installing and run- 
ning third-party packages, like the ones used in the projects in the second 
half of the book. 


The deadsnakes package has been actively maintained for a long time. When newer 
versions of Python come out, you can use these same commands, replacing python3.11 
with the latest version currently available. 


Checking Which Version of Python You're Using 


If you're having any issues running Python or installing additional pack- 
ages, it can be helpful to know exactly which version of Python you're using. 
You may have multiple versions of Python installed and not be clear about 
which version is currently being used. 

Issue the following command in a terminal: 


$ python --version 
Python 3.11.0 


This tells you exactly which version the command python is currently 
pointing to. The shorter command python -V will give the same output. 


Python Keywords and Built-in Functions 
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Python comes with its own set of keywords and built-in functions. It's impor- 
tant to be aware of these when you're naming things in Python: your names 
cannot be the same as these keywords and shouldn't be the same as the 
function names, or you'll overwrite the functions. 

In this section, we'll list Python's keywords and built-in function names, 
so you'll know which names to avoid. 


Python Keywords 


Each of the following keywords has a specific meaning, and you'll see an 
error if you try to use any of them as a variable name. 


False await else import pass 
None break except in raise 
True class finally is return 
and continue for lambda try 
as def from nonlocal while 
assert del global not with 
async elif if or yield 


Python Built-in Functions 


You won't get an error if you use one of the following readily available built- 
in functions as a variable name, but you'll override the behavior of that 
function: 


abs() complex() hash() min() slice() 
aiter() delattr() help() next() sorted() 
all() dict() hex() object() staticmethod() 
any() dir() id() oct() str() 
anext() divmod() input() open() sum() 
ascii() enumerate() int() ord() super() 
bin() eval() isinstance()  pow() tuple() 
bool() exec() issubclass()  print() type() 
breakpoint() filter() iter() property() — vars() 
bytearray() float() len() range() zip() 
bytes() format() list() repr() . import () 
callable() frozenset() locals() reversed() 

chr() getattr() map() round() 

classmethod() globals() max() set() 

compile() hasattr() memoryview() | setattr() 
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TEXT EDITORS AND IDES 


Programmers spend a lot of time writing, 
reading, and editing code, and using a text 


editor or an IDE (integrated development 
environment) to make this work as efficient as 

possible is essential. A good editor will do simple tasks, 

like highlighting your code’s structure so you can catch 


common bugs as you're working. But it won't do so 


much that it distracts you from your thinking. Editors also have useful fea- 
tures like automatic indenting, markers to show appropriate line length, 
and keyboard shortcuts for common operations. 

An IDE is a text editor with a number of other tools included, like inter- 
active debuggers and code introspection. An IDE examines your code as 
you enter it and tries to learn about the project you're building. For example, 
when you start typing the name of a function, an IDE might show you all 
the arguments that function accepts. This behavior can be very helpful 
when everything works and you understand what you're seeing. But it can 
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also be overwhelming as a beginner and difficult to troubleshoot when you 
aren’t sure why your code isn’t working in the IDE. 

These days, the lines have blurred between text editors and IDEs. Most 
popular editors have some features that used to be exclusive to IDEs. Likewise, 
most IDEs can be configured to run in a lighter mode that’s less distracting as 
you work, but lets you use the more advanced features when you need them. 

If you already have an editor or IDE installed that you like, and if it’s 
already configured to work with a recent version of Python that’s installed 
on your system, then I encourage you to stick with what you already know. 
Exploring different editors can be fun, but it’s also a way to avoid the work 
of learning a new language. 

If you don’t already have an editor or IDE installed, I recommend 
VS Code for a number of reasons: 


e It’s free, and it’s released under an open source license. 
e It can be installed on all major operating systems. 


e = It’s beginner-friendly but also powerful enough that many professional 
programmers use it as their main editor. 


e Itfinds the versions of Python you have installed, and it typically does 
not require any configuration to run your first programs. 


e It has an integrated terminal, so your output appears in the same win- 
dow as your code. 


e A Python extension is available that makes the editor highly efficient 
for writing and maintaining Python code. 


e It’s highly customizable, so you can tune it to match the way you work 
with code. 


In this appendix, you'll learn how to start configuring VS Code so that 
it works well for you. You'll also learn some shortcuts that let you work more 
efficiently. Being a fast typist is not as important as many people think in 
programming, but understanding your editor and knowing how to use it 
efficiently is quite helpful. 

With all that said, VS Code doesn't work for everyone. If it doesn't work 
well on your system for some reason, or if it's distracting you as you work, 
there are a number of other editors that you might find more appealing. 
This appendix includes a brief description of some of the other editors and 
IDEs you should consider. 


Working Efficiently with VS Code 
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In Chapter 1, you installed VS Code and added the Python extension as 
well. This section will show you some further configurations you can make, 
plus shortcuts for working efficiently with your code. 


Configuring VS Code 


There are a few ways to change the default configuration settings for VS Code. 
Some changes can be made through the interface, and some will require 


changes in configuration files. These changes will sometimes take effect for 
everything you do in VS Code, while others will affect only the files within 
the folder that contains the configuration file. 

For example, if you have a configuration file in your python_work folder, 
those settings will only affect the files in that folder (and its subfolders). 
This is a good feature, because it means you can have project-specific set- 
tings that override your global settings. 


Using Tabs and Spaces 


If you use a mix of tabs and spaces in your code, it can cause problems in 
your programs that are difficult to diagnose. When working in a .py file with 
the Python extension installed, VS Code is configured to insert four spaces 
whenever you press the TAB key. If you’re writing only your own code and 
you have the Python extension installed, you'll likely never have an issue 
with tabs and spaces. 

However, your installation of VS Code may not be configured correctly. 
Also, at some point, you may end up working on a file that has only tabs or a 
mix of tabs and spaces. If you suspect any issue with tabs and spaces, look at 
the status bar at the bottom of the VS Code window and click either Spaces 
or Tab Size. A drop-down menu will appear that lets you switch between 
using tabs and using spaces. You can also change the default indentation 
level and convert all indentation in the file to either tabs or spaces. 

If you're looking at some code and you're not sure whether the indenta- 
tion consists of tabs or spaces, highlight several lines of code. This will make 
the invisible whitespace characters visible. Each space will show up as a dot, 
and each tab will show up as an arrow. 


In programming, spaces are preferred over tabs because spaces can be interpreted 
unambiguously by all tools that work with a code file. The width of tabs can be inter- 
preted differently by different tools, which leads to errors that can be extremely difficult 
to diagnose. 


Changing the Color Theme 


VS Code uses a dark theme by default. If you want to change this, click File 
(Code in the menu bar on macOS), then click Preferences and choose 
Color Theme. A drop-down list will appear, and it will let you choose a 
theme that works well for you. 


Setting the Line Length Indicator 


Most editors allow you to set up a visual cue, usually a vertical line, to show 
where your lines should end. In the Python community, the convention is to 
limit lines to 79 characters or less. 

To set this feature, click Code and then Preferences, and then choose 
Settings. In the dialog that appears, enter rulers. You'll see a setting for 
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settings.json 


launch. json 
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Editor: Rulers; click the link labeled Edit in settings.json. In the file that 
appears, add the following to the editor.rulers setting: 


80, 


This will add a vertical line in the editing window at the 80-character 
position. You can have more than one vertical line; for example, if you want 
an additional line at 120 characters, the value for your setting would be 
[80, 120]. If you don't see the vertical lines, make sure you saved the settings 
file; you may also need to quit and reopen VS Code for the changes to take 
effect on some systems. 


Simplifying the Output 

By default, VS Code shows the output of your programs in an embedded 
terminal window. This output includes the commands that are being used 
to run the file. For many situations, this is ideal, but it might be more dis- 
tracting than you want when you're first learning Python. 

To simplify the output, close all the tabs that are open in VS Code and 
then quit VS Code. Launch VS Code again and open the folder that con- 
tains the Python files you’re working on; this could just be the python_work 
folder where hello_world.py is saved. 

Click the Run/Debug icon (which looks like a triangle with a small 
bug), and then click Create a launch.json File. Select the Python options in 
the prompts that appear. In the /aunch.json file that opens, make the follow- 
ing change: 


"console": "internalConsole", 


Here, we're changing the console setting from integratedTerminal to 
internalConsole. After saving the settings file, open a .py file such as hello 
_world.py, and run it by pressing CTRL-F5. In the output pane of VS Code, click 
Debug Console if it's not already selected. You should see only your program's 
output, and the output should be refreshed every time you run a program. 


The Debug Console is read-only. It won't work for files that use the input() function, 
which you'll start using in Chapter 7. When you need to run these programs, you can 
either change the console setting back to the default integratedTerminal, or you can 
run these programs in a separate terminal window as described in “Running Python 
Programs from a Terminal" on page 11. 


Exploring Further Customizations 


You can customize VS Code in many ways to help you work more efficiently. To 
start exploring the customizations available, click Code and then Preferences, 
and then choose Settings. You'll see a list titled Commonly Used; click any of 
the subheadings to see some common ways you can modify your installation 
of VS Code. Take some time to see if there are any that make VS Code work 
better for you, but don’t get so lost in configuring your editor that you put off 
learning how to use Python! 


VS Code Shortcuts 


All editors and IDEs offer efficient ways to do common tasks that everyone 
needs to do when writing and maintaining code. For example, you can eas- 
ily indent a single line of code or an entire block of code; you can just as 
easily move a block of lines up or down in a file. 

There are too many shortcuts to describe fully here. This section will 
share just a few that you'll likely find helpful as you're writing your first 
Python files. If you end up using a different editor than VS Code, make sure 
you learn how to do these same tasks efficiently in the editor you've chosen. 


Indenting and Unindenting Code Blocks 


To indent an entire block of code, highlight it and press CTRL-], or 3-] on 
macOS. To unindent a block of code, highlight it and press CTRL-[, or 88-[ 
on macOS. 


Commenting Out Blocks of Code 


To temporarily disable a block of code, you can highlight the block and 
comment it so Python will ignore it. Highlight the section of code you want 
to ignore and press CTRL-/, or 88-/ on macOS. The selected lines will 
be commented out with a hash mark (£) indented at the same level as the 
line of code, to indicate these are not regular comments. When you want 
to uncomment the block of code, highlight the block and reissue the same 
command. 


Moving Lines Up or Down 


As your programs grow more complex, you may find that you want to move a 
block of code up or down within a file. To do so, highlight the code you want 
to move and press ALT-up arrow, or Option-up arrow on macOS. The same 
key combination with the down arrow will move the block down in the file. 

If you're moving a single line up or down, you can click anywhere in 
that line; you don't need to highlight the whole line to move it. 


Hiding the File Explorer 


The integrated file explorer in VS Code is really convenient. However, it can 
be distracting when you're writing code and can take up valuable space on 
a smaller screen. The command CTRL-B, or 88-B on macOS, toggles the 
visibility of the file explorer pane. 
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Finding Additional Shortcuts 


Working efficiently in an editing environment takes practice, but it also 
takes intention. When you're learning to work with code, try to notice the 
things you do repeatedly. Any action you take in your editor likely has a 
shortcut; if you're clicking menu items to carry out editing tasks, look for 
the shortcuts for those actions. If you're switching between your keyboard 
and mouse frequently, look for the navigation shortcuts that keep you from 
reaching for your mouse so often. 

You can see all the keyboard shortcuts in VS Code by clicking Code and 
then Preferences, and then choosing Keyboard Shortcuts. You can use the 
search bar to find a particular shortcut, or you can scroll through the list to 
find shortcuts that might help you work more efficiently. 

Remember, it's better to focus on the code that you're working on, and 
avoid spending too much time on the tools you're using. 


Other Text Editors and IDEs 
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You'll hear about and see people using a number of other text editors. Most 
of them can be configured to help you in the same way you've customized 
VS Code. Here's a small selection of text editors you might hear about. 


IDLE 


IDLE s a text editor that's included with Python. It's a little less intuitive to 
work with than other, more modern editors. However, you'll see references 
to itin other tutorials aimed at beginners, so you might want to give it a try. 


Geany 


Geany is a simple text editor that displays all of your output in a separate ter- 
minal window, which helps you become comfortable using terminals. Geany 
has a very minimalist interface, but it's powerful enough that a significant 
number of experienced programmers still use it. 

If you find VS Code too distracting and full of too many features, con- 
sider using Geany instead. 


Sublime Text 


Sublime Text is another minimalist editor that you should consider using if 
you find VS Code too busy. Sublime Text has a really clean interface and is 
known for working well even on very large files. It’s an editor that will get 
out of your way and let you focus on the code you're writing. 

Sublime Text has an unlimited free trial, but it's not free or open 
source. If you decide you like it and can afford to purchase a full license, 
you should do so. The purchase is a one-time fee; it's not a software 
subscription. 


Emacs and Vim 


Emacs and Vim are two popular editors favored by many experienced pro- 
grammers, because they’re designed so you can use them without your 
hands ever having to leave the keyboard. This makes writing, reading, and 
modifying code very efficient, once you learn how the editor works. It also 
means both editors have a fairly steep learning curve. Vim is included on 
most Linux and macOS machines, and both Emacs and Vim can be run 
entirely inside a terminal. For this reason, they’re often used to write code 
on servers through remote terminal sessions. 

Programmers will often recommend that you give them a try, but many 
proficient programmers forget how much new programmers are already 
trying to learn. It’s good to be aware of these editors, but you should hold 
off on using them until you’re comfortable working with code in a more 
user-friendly editor that lets you focus on learning to program, rather than 
learning to use an editor. 


PyCharm 


PyCharm is a popular IDE among Python programmers because it was built 
to work specifically with Python. The full version requires a paid subscrip- 
tion, but a free version called the PyCharm Community Edition is also 
available, and many developers find it useful. 

If you try PyCharm, be aware that, by default, it sets up an isolated envi- 
ronment for each of your projects. This is usually a good thing, but it can 
lead to unexpected behavior if you don’t understand what it’s doing for you. 


Jupyter Notebooks 


Jupyter Notebook is a different kind of tool than traditional text editors or 
IDEs, in that it’s a web app primarily built of blocks; each block is either a 
code block or a text block. The text blocks are rendered in Markdown, so 
you can include simple formatting in your text blocks. 

Jupyter Notebooks were developed to support the use of Python in 
scientific applications, but they have since expanded to become useful in 
a wide variety of situations. Rather than just writing comments inside a .py 
file, you can write clear text with simple formatting, such as headers, bul- 
leted lists, and hyperlinks in between sections of code. Every code block can 
be run independently, allowing you to test small pieces of your program, or 
you can run all the code blocks at once. Each code block has its own output 
area, and you can toggle the output areas on or off as needed. 

Jupyter Notebooks can be confusing at times because of the interactions 
between different cells. If you define a function in one cell, that function is 
available to other cells as well. This is beneficial most of the time, but it can 
be confusing in longer notebooks and if you don’t fully understand how the 
Notebook environment works. 

If you're doing any scientific or data-focused work in Python, you'll 
almost certainly see Jupyter Notebooks at some point. 
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GETTING HELP 


Everyone gets stuck at some point when 
they’re learning to program. So, one of 
the most important skills to learn as a pro- 
grammer is how to get unstuck efficiently. This 
appendix outlines several ways to help you get going 
again when programming gets confusing. 


First Steps 


When you're stuck, your first step should be to assess your situation. Before 


you ask for help from anyone else, answer the following three questions 
clearly: 


e What are you trying to do? 
e What have you tried so far? 


e What results have you been getting? 
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Make your answers as specific as possible. For the first question, explicit 
statements like “I’m trying to install the latest version of Python on my new 
Windows laptop” are detailed enough for others in the Python community 
to help you. Statements like “I’m trying to install Python” don’t provide 
enough information for others to offer much help. 

Your answer to the second question should provide enough detail so 
you won't be advised to repeat what you've already tried: “I went to hitps:// 
fython.org/downloads and clicked the Download button for my system. Then 
Iran the installer" is more helpful than *I went to the Python website and 
downloaded something." 

For the third question, it's helpful to know the exact error messages 
you received, so you can use them to search online for a solution or provide 
them when asking for help. 

Sometimes, just answering these three questions before you ask for help 
from others allows you to see something you're missing, and helps get you 
unstuck without having to go any further. Programmers even have a name 
for this: rubber duck debugging. The idea is that if you clearly explain your 
situation to a rubber duck (or any inanimate object) and ask it a specific 
question, you'll often be able to answer your own question. Some program- 
ming teams even keep a real rubber duck around to encourage people to 
"talk to the duck." 


Try It Again 

Just going back to the start and trying again can be enough to solve many 
problems. Say you're trying to write a for loop based on an example in this 
book. You might have only missed something simple, like a colon at the end 
of the for line. Going through the steps again might help you avoid repeat- 
ing the same mistake. 


Take a Break 


If you've been working on the same problem for a while, taking a break is 
one of the best tactics you can try. When we work on the same task for long 
periods of time, our brains start to zero in on only one solution. We lose 
sight of the assumptions we've made, and taking a break helps us get a fresh 
perspective on the problem. It doesn't need to be a long break, just some- 
thing that gets you out of your current mindset. If you've been sitting for a 
long time, do something physical: take a short walk, go outside for a bit, or 
perhaps drink a glass of water or eat a light snack. 

If you're getting frustrated, it might be worth putting your work away 
for the day. A good night's sleep almost always makes a problem more 
approachable. 


Refer to This Book's Resources 


The online resources for this book, available at Attps:/ehmatthes. github. io/pcc_3e, 
include a number of helpful sections about setting up your system and work- 
ing through each chapter. If you haven't done so already, take a look at these 
resources and see if there's anything that helps your situation. 


Searching Online 


Chances are good that someone else has had the same problem you're 
having and has written about it online. Good searching skills and specific 
inquiries will help you find existing resources to solve the issue you're fac- 
ing. For example, if you're struggling to install the latest version of Python 
on a new Windows system, searching for install python windows and limiting 
the results to resources from the last year might direct you to a clear answer. 

Searching the exact error message can be extremely helpful too. For 
example, say you get the following error when you try to run a Python pro- 
gram from a terminal on a new Windows system: 


> python hello world.py 
Python was not found; run without arguments to install from the Microsoft 
Store... 


Searching for the full phrase, *Python was not found; run without argu- 
ments to install from the Microsoft Store," will probably yield some good 
advice. 

When you start searching for programming-related topics, a few sites 
will appear repeatedly. I'll describe some of these sites briefly, so you'll 
know how helpful they're likely to be. 


Stack Overflow 


Stack Overflow (https://stackoverflow.com) is one of the most popular question- 
and-answer sites for programmers, and it will often appear in the first page 
of results on Python-related searches. Members post questions when they're 
stuck, and other members try to give helpful responses. Users can vote for 
the responses they find most helpful, so the best answers are usually the 
first ones you'll find. 

Many basic Python questions have very clear answers on Stack Overflow, 
because the community has refined them over time. Users are encouraged 
to post updates, too, so responses tend to stay relatively current. At the 
time of this writing, almost two million Python-related questions have been 
answered on Stack Overflow. 

There's one expectation you should be aware of before posting on 
Stack Overflow. Questions are meant to be the shortest example of the 
kind of issue you're facing. If you post 5-20 lines of code that generate the 
error you're facing, and if you address the questions mentioned in *First 
Steps" on page 477 earlier in this appendix, someone will probably help 
you. If you share a link to a project with multiple large files, people will be 
very unlikely to help. There's a great guide to writing up a good question 
at hitps://stackoverflow.com/help/how-to-ask. The suggestions in this guide are 
applicable to getting help in any community of programmers. 


The Official Python Documentation 


The official Python documentation (Attps://docs.bython.org) is a bit more 
hit-or-miss for beginners, because its purpose is more to document the 
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language than to provide explanations. The examples in the official docu- 
mentation should work, but you might not understand everything shown. 
Still, it’s a good resource to check when it comes up in your searches, and it 
will become more useful to you as you continue building your understand- 
ing of Python. 


Official Library Documentation 


If you're using a specific library, such as Pygame, Matplotlib, or Django, 
links to the official documentation for it will often appear in searches. For 
example, Attps://docs.djangoproject.com is very helpful when working with 
Django. If you’re planning to work with any of these libraries, it’s a good 
idea to become familiar with their official documentation. 


r/learnpython 


Reddit is made up of a number of subforums called subreddits. The v/learnpython 
subreddit (Attps://reddit. com/r/learnpython) is very active and supportive. You 
can read others’ questions and post your own as well. You will often get mul- 
tiple perspectives about the questions you raise, which can be really helpful 
in gaining a deeper understanding of the topic you're working on. 


Blog Posts 


Many programmers maintain blogs and share posts about the parts of the 
language they’re working with. You should look for a date on the blog posts 
you find, to see how applicable the information is likely to be for the version 
of Python you're using. 


Discordis an online chat environment with a Python community where you 
can ask for help and follow Python-related discussions. 

To check it out, head to hitps://pythondiscord.com and click the Discord 
link at the upper right. If you already have a Discord account, you can log in 
with your existing account. If you don't have an account, enter a username 
and follow the prompts to complete your Discord registration. 

If this is your first time visiting the Python Discord, you'll need to accept 
the rules for the community before participating fully. Once you've done 
that, you can join any of the channels that interest you. If you're looking for 
help, be sure to post in one of the Python Help channels. 


Slack 


Slack is another online chat environment. It is often used for internal com- 
pany communications, but there are also many public groups you can join. 
If you want to check out Python Slack groups, start with hétps://pyslackers.com. 
Click the Slack link at the top of the page, then enter your email address to 
get an invitation. 

Once you're in the Python Developers workspace, you'll see a list of 
channels. Click Channels and then choose the topics that interest you. You 
might want to start with the #help and #django channels. 
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USING GIT FOR 
VERSION CONTROL 


Version control software allows you to take 
snapshots of a project whenever it’s in a 


working state. When you make changes to a 
project—for example, when you implement a 

new feature—you can go back to a previous working 

state if the project’s current state isn’t functioning well. 


Using version control software gives you the freedom to work on improve- 
ments and make mistakes without worrying about ruining your project. This 
is especially critical in large projects, but can also be helpful in smaller proj- 
ects, even when you're working on programs contained in a single file. 

In this appendix, you'll learn to install Git and use it for version control in 
the programs you're working on now. Gitis the most popular version control 
software in use today. Many of its advanced tools help teams collaborate on 
large projects, but its most basic features also work well for solo developers. 
Git implements version control by tracking the changes made to every file in a 
project; if you make a mistake, you can just return to a previously saved state. 
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Installing Git 


Git runs on all operating systems, but there are different approaches to 
installing it on each system. The following sections provide specific instruc- 
tions for each operating system. 

Git is included on some systems by default, and is often bundled with 
other packages that you might have already installed. Before trying to install 
Git, see if it's already on your system. Open a new terminal window and issue 
the command git --version. If you see output listing a specific version num- 
ber, Git is installed on your system. If you see a message prompting you to 
install or update Git, follow the onscreen instructions. 

If you don't see any onscreen instructions and you're using Windows or 
macOS, you can download an installer from hitps://git-scm.com. If you're a 
Linux user with an apt-compatible system, you can install Git with the com- 
mand sudo apt install git. 


Configuring Git 

Git keeps track of who makes changes to a project, even when only one per- 
son is working on the project. To do this, Git needs to know your username 
and email. You must provide a username, but you can make up a fake email 
address: 


$ git config --global user.name "username" 
$ git config --global user.email "username@example.com" 


If you forget this step, Git will prompt you for this information when 
you make your first commit. 

It's also best to set the default name for the main branch in each proj- 
ect. A good name for this branch is main: 


$ git config --global init.defaultBranch main 


This configuration means that each new project you use Git to manage 
will start out with a single branch of commits called main. 


Making a Project 


Let's make a project to work with. Create a folder somewhere on your sys- 
tem called git practice. Inside the folder, make a simple Python program: 


hello git.py  print("Hello Git world!") 


We'll use this program to explore Git's basic functionality. 


Ignoring Files 
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Files with the extension .pyc are automatically generated from .py files, so we 
don't need Git to keep track of them. These files are stored in a directory 


file called .gitignore—with a dot at the beginning of the filename and no file 
extension—and add the following line to it: 


called __pycache__. To tell Git to ignore this directory, make a special 


.gitignore — pycache / 


This file tells Git to ignore any file in the __pycache__ directory. Using a 
.gitignore file will keep your project clutter-free and easier to work with. 

You might need to modify your file browser’s settings so hidden files 
(files whose names begin with a dot) will be shown. In Windows Explorer, 
check the box in the View menu labeled Hidden Items. On macOS, press 
38-SHIFT-. (dot). On Linux, look for a setting labeled Show Hidden Files. 


If you’re on macOS, add one more line to .gitignore. Add the name .DS_Store; these 
are hidden files that contain information about each directory on macOS, and they 
will clutter up your project if you don't add them to .gitignore. 


Initializing a Repository 


Now that you have a directory containing a Python file and a .gitignore file, 
you can initialize a Git repository. Open a terminal, navigate to the gi practice 
folder, and run the following command: 


git practice$ git init 
Initialized empty Git repository in git practice/.git/ 
git practice$ 


The output shows that Git has initialized an empty repository in 
gil, practice. A repository is the set of files in a program that Git is actively 
tracking. All the files Git uses to manage the repository are located in the 
hidden directory .git, which you won't need to work with at all. Just don't 
delete that directory, or you'll lose your project's history. 


Checking the Status 


Before doing anything else, let's look at the project's status: 


git practice$ git status 
6 On branch main 
No commits yet 


@ Untracked files: 
(use "git add «file»..." to include in what will be committed) 
.gitignore 
hello git.py 


© nothing added to commit but untracked files present (use "git add" to track) 
git practice$ 
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In Git, a branch is a version of the project you’re working on; here you 
can see that we're on a branch named main 6. Each time you check your 
project's status, it should show that you're on the branch main. You then see 
that we're about to make the initial commit. A commit is a snapshot of the 
project at a particular point in time. 

Git informs us that untracked files are in the project @, because we 
haven't told it which files to track yet. Then we're told that there's nothing 
added to the current commit, but untracked files are present that we might 
want to add to the repository 9. 


Adding Files to the Repository 


Let's add the two files to the repository and check the status again: 


@ git practice$ git add . 
@ git practice$ git status 
On branch main 
No commits yet 


Changes to be committed: 
(use "git rm --cached «file»..." to unstage) 
e new file: .gitignore 
new file: hello git.py 


git practice$ 


The command git add . adds to the repository all files within a project 
that aren't already being tracked 6, as long as they're not listed in .gitignore. 
It doesn't commit the files; it just tells Git to start paying attention to them. 
When we check the status of the project now, we can see that Git recognizes 
some changes that need to be committed @. The label new file means these 
files were newly added to the repository 9. 


Making a Commit 


Let's make the first commit: 


@ git practice$ git commit -m "Started project." 
@ [main (root-commit) ceai3dd] Started project. 
© 2 files changed, 5 insertions(+) 

create mode 100644 .gitignore 

create mode 100644 hello git.py 
@ git practice$ git status 

On branch main 

nothing to commit, working tree clean 

git practice$ 


We issue the command git commit -m " message" € to make a snapshot of 
the project. The -m flag tells Git to record the message that follows (Started 
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project.) in the project’s log. The output shows that we’re on the main 
branch @ and that two files have changed ®. 

When we check the status now, we can see that we’re on the main branch, 
and we have a clean working tree O. This is the message you should see 
each time you commit a working state of your project. If you get a different 
message, read it carefully; it's likely you forgot to add a file before making a 
commit. 


Checking the Log 


Git keeps a log of all commits made to the project. Let's check the log: 


git practice$ git log 

commit cea13ddc51b885d05a410201a54faf20e0d2e246 (HEAD -» main) 
Author: eric <eric@example.com> 

Date: Mon Jun 6 19:37:26 2022 -0800 


Started project. 
git practice$ 


Each time you make a commit, Git generates a unique, 40-character 
reference ID. It records who made the commit, when it was made, and the 
message recorded. You won't always need all of this information, so Git pro- 
vides an option to print a simpler version of the log entries: 


git practice$ git log --pretty-oneline 
cea13ddc51b885d052410201a54faf20e0d2e246 (HEAD -> main) Started project. 
git practice$ 


The --pretty=oneline flag provides the two most important pieces of 
information: the reference ID of the commit and the message recorded for 
the commit. 


The Second Commit 


To see the real power of version control, we need to make a change to the proj- 
ect and commit that change. Here we'll just add another line to hello_git.py: 


hello git.py ello ( NO ) 
print("Hello everyone.") 


When we check the status of the project, we'll see that Git has noticed 
the file that changed: 


git practice$ git status 
€ On branch main 
Changes not staged for commit: 
(use "git add «file»..." to update what will be committed) 
(use "git restore «file»..." to discard changes in working directory) 
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O modified: hello git.py 


© no changes added to commit (use "git add" and/or "git commit -a") 


git practice$ 


We see the branch we're working on 6, the name of the file that was 
modified @, and that no changes have been committed 9. Let's commit 
the change and check the status again: 


@ git practice$ git commit -am "Extended greeting." 


[main 945fa13] Extended greeting. 
1 file changed, 1 insertion(+), 1 deletion(-) 


@ git practice$ git status 


On branch main 
nothing to commit, working tree clean 


© git practice$ git log --pretty-oneline 


945fa13af128a266d0114eebb7a3276f7d58ecd2 (HEAD -» main) Extended greeting. 
cea13ddc51b885d05a410201a54faf20e0d2e246 Started project. 
git practice$ 


We make a new commit, passing the -am flags when we use the command 
git commit €. The -a flag tells Git to add all modified files in the repository 
to the current commit. (If you create any new files between commits, reissue 
the git add . command to include the new files in the repository.) The -m flag 
tells Git to record a message in the log for this commit. 

When we check the project's status, we see that we once again have a 
clean working tree 6. Finally, we see the two commits in the log 9. 


Abandoning Changes 


hello git.py 
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Now let's look at how to abandon a change and go back to the previous 
working state. First, add a new line to hello_git.py: 


print("Oh no, I broke the project!") 


Save and run this file. 
We check the status and see that Git notices this change: 


git practice$ git status 
On branch main 
Changes not staged for commit: 
(use "git add «file»..." to update what will be committed) 
(use "git restore «file»..." to discard changes in working directory) 


modified: hello git.py 


no changes added to commit (use "git add" and/or "git commit -a" 
git practice$ 


Git sees that we modified hello_git.py €, and we can commit the change 
if we want to. But this time, instead of committing the change, we’ll go back 
to the last commit when we knew our project was working. We won't do any- 
thing to hello git.py: we won't delete the line or use the Undo feature in the 
text editor. Instead, enter the following commands in your terminal session: 


git practice$ git restore . 

git practice$ git status 

On branch main 

nothing to commit, working tree clean 
git practice$ 


The command git restore filename allows you to abandon all changes 
since the last commit in a specific file. The command git restore . aban- 
dons all changes made in all files since the last commit; this action restores 
the project to the last committed state. 

When you return to your text editor, you'll see that hello_git.py has 
changed back to this: 


Although going back to a previous state might seem trivial in this simple 
project, if we were working on a large project with dozens of modified files, 
all the files that had changed since the last commit would be restored. This 
feature is incredibly useful: you can make as many changes as you want 
when implementing a new feature, and if they don't work, you can discard 
them without affecting the project. You don't have to remember those 
changes and manually undo them. Git does all of that for you. 


You might have to refresh the file in your editor to see the restored. version. 


Checking Out Previous Commits 


You can revisit any commit in your log, using the checkout command, by 
using the first six characters of a reference ID. After checking out and 
reviewing an earlier commit, you can return to the latest commit or aban- 
don your recent work and pick up development from the earlier commit: 


git practice$ git log --pretty-oneline 
945fa13af128a266d0114eebb7a3276f7d58ecd2 (HEAD -» main) Extended greeting. 
cea13ddc51b885d05a410201a54faf20e0d2e246 Started project. 

git practice$ git checkout cea13d 

Note: switching to 'ceai13d'. 


@ You are in 'detached HEAD' state. You can look around, make experimental 


changes and commit them, and you can discard any commits you make in this 
state without impacting any branches by switching back to a branch. 
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If you want to create a new branch to retain commits you create, you may 
do so (now or later) by using -c with the switch command. Example: 


git switch -c <new-branch-name> 
@ Or undo this operation with: 
git switch - 
Turn off this advice by setting config variable advice.detachedHead to false 


HEAD is now at cea13d Started project. 
git practice$ 


When you check out a previous commit, you leave the main branch and 
enter what Git refers to as a detached HEAD state €. HEAD is the current 
committed state of the project; you're detached because you've left a named 
branch (main, in this case). 

To get back to the main branch, you follow the suggestion to undo the 
previous operation: 


git practice$ git switch - 

Previous HEAD position was cea13d Started project. 
Switched to branch 'main' 

git practice$ 


This command brings you back to the main branch. Unless you want 
to work with some more advanced features of Git, it's best not to make 
any changes to your project when you've checked out a previous commit. 
However, if you're the only one working on a project and you want to dis- 
card all of the more recent commits and go back to a previous state, you 
can reset the project to a previous commit. Working from the main branch, 
enter the following: 


@ git practice$ git status 
On branch main 
nothing to commit, working directory clean 

@ git practice$ git log --pretty-oneline 
945fa13af128a266d0114eebb7a3276f7d58ecd2 (HEAD -» main) Extended greeting. 
cea13ddc51b885d05a410201a54faf20e0d2e246 Started project. 

© git practice$ git reset --hard cea13d 
HEAD is now at ceai3dd Started project. 

O git practice$ git status 
On branch main 
nothing to commit, working directory clean 

© git practice$ git log --pretty-oneline 
cea13ddc51b885d05a410201a54faf20e0d2e246 (HEAD -> main) Started project. 
git practice$ 


We first check the status to make sure we're on the main branch 6. 
When we look at the log, we see both commits 6. We then issue the git 
reset --hard command with the first six characters of the reference ID of 


the commit we want to go back to permanently 6. We check the status 
again and see we're on the main branch with nothing to commit @. When 
we look at the log again, we see that we're at the commit we wanted to start 
over from 8. 


Deleting the Repository 


Sometimes you'll mess up your repository's history and won't know how to 
recover it. If this happens, first consider asking for help using the approaches 
discussed in Appendix C. If you can't fix it and you're working on a solo proj- 
ect, you can continue working with the files but get rid of the project's history 
by deleting the .git directory. This won't affect the current state of any of the 
files, but it will delete all commits, so you won't be able to check out any other 
states of the project. 

To do this, either open a file browser and delete the .git repository or 
delete it from the command line. Afterward, you'll need to start over with a 
fresh repository to start tracking your changes again. Here's what this entire 
process looks like in a terminal session: 


@ git practice$ git status 

On branch main 

nothing to commit, working directory clean 
© git practice$ rm -rf .git/ 
© git practice$ git status 

fatal: Not a git repository (or any of the parent directories): .git 
O git practice$ git init 

Initialized empty Git repository in git practice/.git/ 
© git practice$ git status 

On branch main 

No commits yet 


Untracked files: 
(use "git add «file»..." to include in what will be committed) 
.gitignore 
hello git.py 


nothing added to commit but untracked files present (use "git add" to track) 
@ git practice$ git add . 
git practice$ git commit -m "Starting over." 
[main (root-commit) 14ed9db] Starting over. 
2 files changed, 5 insertions(+) 
create mode 100644 .gitignore 
create mode 100644 hello git.py 
@ git practice$ git status 
On branch main 
nothing to commit, working tree clean 
git practice$ 


We first check the status and see that we have a clean working direc- 
tory 6. Then we use the command rm -rf .git/ to delete the .git directory 
(del .git on Windows) 9. When we check the status after deleting the .git 
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folder, we're told that this is not a Git repository 9. All the information Git 
uses to track a repository is stored in the .git folder, so removing it deletes 
the entire repository. 

We're then free to use git init to start a fresh repository @. Checking 
the status shows that we're back at the initial stage, awaiting the first com- 
mit ©. We add the files and make the first commit O. Checking the status 
now shows us that we're on the new main branch with nothing to commit 9. 


Using version control takes a bit of practice, but once you start using it, 
you'll never want to work without it again. 
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TROUBLESHOOTING 
DEPLOYMENTS 


Deploying an app is tremendously satisfy- 
ing when it works, especially if you’ve never 
done it before. However, there are many 
obstacles that can arise in the deployment pro- 
cess, and unfortunately, some of these issues can be 
difficult to identify and address. This appendix will 
help you understand modern approaches to deploy- 
ment and give you specific ways to troubleshoot the 


deployment process when things aren’t working. 

If the additional information in this appendix isn’t enough to help you 
get through the deployment process successfully, see the online resources at 
https://ehmatthes.github.io/pcc_3e; the updates there will almost certainly help 
you carry out a successful deployment. 
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Understanding Deployments 


When you're trying to troubleshoot a particular deployment attempt, it's 
helpful to have a clear understanding of how a typical deployment works. 
Deployment refers to the process of taking a project that works on your local 
system, and copying that project to a remote server in a way that allows it 
to respond to requests from any user on the internet. The remote environ- 
ment differs from a typical local system in a number of important ways: it’s 
probably not the same operating system (OS) as the one you're using, and 
it’s most likely one of many virtual servers on a single physical server. 

When you deploy a project, or push it to the remote server, the following 
steps need to be taken: 


e Create a virtual server on a physical machine at a datacenter. 
e Establish a connection between the local system and the remote server. 
e Copy the project's code to the remote server. 


e Identify all of the project's dependencies and install them on the 
remote server. 


e Setup a database and run any existing migrations. 


e Copy static files (CSS, JavaScript files, and media files) to a place where 
they can be served efficiently. 


e Start a server to handle incoming requests. 


e Start routing incoming requests to the project, once it's ready to handle 
requests. 


When you consider all that goes into a deployment, it's no wonder 
deployments often fail. Fortunately, once you gain an understanding of 
what should be happening, you'll stand a better chance of identifying what 
went wrong. If you can identify what went wrong, you might be able to iden- 
tify a fix that will make the next deployment attempt successful. 

You can develop locally on one kind of OS and push to a server running 
a different OS. It's important to know what kind of system you're pushing to, 
because that can inform some of your troubleshooting work. At the time of 
this writing, a basic remote server on Platform.sh runs Debian Linux; most 
remote servers are Linux-based systems. 


Basic Troubleshooting 
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Some troubleshooting steps are specific to each OS, but we'll get to that in 
a moment. First, let's consider the steps everyone should try when trouble- 
shooting a deployment. 

Your best resource is the output generated during the attempted 
push. This output can look intimidating; if you're new to deploying apps, 
it can look highly technical, and there's usually a lot of it. The good news 
is you don't need to understand everything in the output. You should have 
two goals when skimming log output: identify any deployment steps that 
worked, and identify any steps that didn't. If you can do this, you might be 


able to figure out what to change in your project, or in your deployment 
process, to make your next push successful. 


Follow Onscreen Suggestions 


Sometimes, the platform you're pushing to will generate a message that has 
a clear suggestion for how to address the issue. For example, here’s the mes- 
sage you'll see if you create a Platform.sh project before initializing a Git 
repository, and then try to push the project: 


$ platform push 
@ Enter a number to choose a project: 
[0] 11 project (votohz4451jyg) 
>0 


@  [RootNotFoundException] 
Project root not found. This can only be run from inside a project 
directory. 


© To set the project for this Git repository, run: 
platform project:set-remote [id] 


We're trying to push a project, but the local project hasn't been associ- 
ated with a remote project yet. So, the Platform.sh CLI asks which remote 
project we want to push to 6. We enter 0, to select the only project listed. 
But next, we see a RootNotFoundException @. This happens because Platform.sh 
looks for a .git directory when it inspects the local project, to figure out 
how to connect the local project with the remote project. In this case, since 
there was no .git directory when the remote project was created, that con- 
nection was never established. The CLI suggests a fix 9; it's telling us that 
we can specify the remote project that should be associated with this local 
project, using the project:set-remote command. 

Let's try this suggestion: 


$ platform project:set-remote votohz445ljyg 
Setting the remote project for this repository to: 11 project (votohz4451jyg) 


The remote project for this repository is 
now set to: 11 project (votohz445ljyg) 


In the previous output, the CLI showed the ID of this remote project, 
votohz4451jyg. So we run the command that's suggested, using this ID, and 
the CLI is able to make the connection between the local project and the 
remote project. 

Now let's try to push the project again: 


$ platform push 

Are you sure you want to push to the main (production) branch? [Y/n] y 
Pushing HEAD to the existing environment main 

--snip-- 


This was a successful push; following the onscreen suggestion worked. 
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You should be careful about running commands that you don’t fully 
understand. However, if you have good reason to believe that a command 
can do little harm, and if you trust the source of the recommendation, it 
might be reasonable to try the suggestions offered by the tools you're using. 


Keep in mind there are individuals who will tell you to run commands that will wipe 
your system or expose your system to remote exploitation. Following the suggestions of 
a tool provided by a company or organization you trust is different from following the 
suggestions of random people online. Anytime you're dealing with remote connections, 
proceed with an abundance of caution. 


Read the Log Output 


As mentioned earlier, the log output that you see when you run a command 
like platform push can be both informative and intimidating. Read through 
the following snippet of log output, taken from a different attempt at using 
platform push, and see if you can spot the issue: 


--snip-- 
Collecting soupsieve==2.3.2.post1 
Using cached soupsieve-2.3.2.posti-py3-none-any.whl (37 kB) 
Collecting sqlparse==0.4.2 
Using cached sqlparse-0.4.2-py3-none-any.whl (42 kB) 
Installing collected packages: platformshconfig, sqlparse,... 
Successfully installed Django-4.1 asgiref-3.5.2 beautifulsoup4-4.11.1... 
W: ERROR: Could not find a version that satisfies the requirement gunicorrn 
W: ERROR: No matching distribution found for gunicorrn 


130 static files copied to '/app/static'. 


Executing pre-flight checks... 
--snip-- 


When a deployment attempt fails, a good strategy is to look through 
the log output and see if you can spot anything that looks like warnings or 
errors. Warnings are fairly common; they're often messages about upcom- 
ing changes in a project's dependencies, to help developers address issues 
before they cause actual failures. 

A successful push may have warnings, but it shouldn't have any errors. 
In this case, Platform.sh couldn't find a way to install the requirement 
gunicorrn. This is a typo in the requirements remote.txt file, which was sup- 
posed to include gunicorn (with one r). It's not always easy to spot the root 
issue in log output, especially when the problem causes a bunch of cascad- 
ing errors and warnings. Just like when reading a traceback on your local 
system, it's a good idea to look closely at the first few errors that are listed, 
and also the last few errors. Most of the errors in between tend to be inter- 
nal packages complaining that something went wrong, and passing mes- 
sages about the error to other internal packages. The actual error we can 
fix is usually one of the first or last errors listed. 


Sometimes, you'll be able to spot the error, and other times, you'll have 
no idea what the output means. It's certainly worth a try, and using log out- 
put to successfully diagnose an error is a tremendously satisfying feeling. As 
you spend more time looking through log output, you'll get better at identi- 
fying the information that's most meaningful to you. 


OS-Specific Troubleshooting 


You can develop on any operating system you like and push to any host 
you like. The tools for pushing projects have developed enough that they'll 
modify your project as needed to run correctly on the remote system. 
However, there are some OS-specific issues that can arise. 

In the Platform.sh deployment process, one of the most likely sources 
of difficulties is installing the CLI. Here's the command to do so: 


$ curl -fsS https://platform.sh/cli/installer | php 


The command starts with curl, a tool that lets you request remote 
resources, accessed through a URL, within a terminal. Here, it's being used 
to download the CLI installer from a Platform.sh server. The -fsS section of 
the command is a set of flags that modify how curl runs. The f flag tells curl 
to suppress most error messages, so the CLI installer can handle them instead 
of reporting them all to you. The s flag tells curl to run silently; it lets the CLI 
installer decide what information to show in the terminal. The S flag tells curl 
to show an error message if the overall command fails. The | php at the end 
of the command tells your system to run the downloaded installer file using a 
PHP interpreter, because the Platform.sh CLI is written in PHP. 

This means your system needs curl and PHP in order to install the 
Platform.sh CLI. To use the CLI, you'll also need Git, and a terminal that 
can run Bash commands. Bash is a language that’s available in most server 
environments. Most modern systems have plenty of room for multiple tools 
like this to be installed. 

The following sections will help you address these requirements for 
your OS. If you don’t already have Git installed, see the instructions for 
installing Git on page 484 in Appendix D and then go to the section here 
that’s applicable to your OS. 


An excellent tool for understanding terminal commands like the one shown here is 
https://explainshell.com. Enter the command you’re trying to understand, and 
the site will show you the documentation for all the parts of your command. Try it 
out with the command used to install the Platform.sh CLI. 


Deploying from Windows 


Windows has seen a resurgence in popularity with programmers in recent 
years. Windows has integrated many different elements of other operating 
systems, providing users with a number of options for how to do local devel- 
opment work and interact with remote systems. 
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One of the most significant difficulties in deploying from Windows is 
that the core Windows operating system is not the same as what a Linux- 
based remote server uses. A base Windows system has a different set of tools 
and languages than a base Linux system, so to carry out deployment work 
from Windows, you’ll need to choose how to integrate Linux-based tool sets 
into your local environment. 


Windows Subsystem for Linux 


One popular approach is to use Windows Subsystem for Linux (WSL), an envi- 
ronment that allows Linux to run directly on Windows. If you have WSL 
set up, using the Platform.sh CLI on Windows becomes as easy as using it 
on Linux. The CLI won’t know it’s running on Windows; it will just see the 
Linux environment you're using it in. 

Setting up WSL is a two-step process: you first install WSL, and then 
choose a Linux distribution to install into the WSL environment. Setting up 
a WSL environment is more than can be described here; if you're interested 
in this approach and don't already have it set up, see the documentation at 
hitps://docs.microsoft.com/en-us/windows/wsl/about. Once you have WSL set up, 
you can follow the instructions in the Linux section of this appendix to con- 
tinue your deployment work. 


Git Bash 


Another approach to building a local environment that you can deploy 
from uses Git Bash, a terminal environment that's compatible with Bash 
but runs on Windows. Git Bash is installed along with Git when you use 
the installer from Attps://git-scm.com. This approach can work, but it isn't as 
streamlined as WSL. In this approach, you'll have to use a Windows termi- 
nal for some steps and a Git Bash terminal for others. 

First you'll need to install PHP. You can do this with XAMPP, a package 
that bundles PHP with a few other developer-focused tools. Go to hitps:// 
apachefriends.org and click the button to download XAMPP for Windows. 
Open the installer and run it; if you see a warning about User Account 
Control (UAC) restrictions, click OK. Accept all of the installer’s defaults. 

When the installer finishes running, you'll need to add PHP to your 
system's path; this will tell Windows where to look when you want to run 
PHP. In the Start menu, enter path and click Edit the System Environment 
Variables; click the button labeled Environment Variables. You should see 
the variable Path highlighted; click Edit under this pane. Click New to add 
a new path to the current list of paths. Assuming you kept the default set- 
tings when running the XAMPP installer, add C:\xampp\php in the box that 
appears, then click OK. When you're finished, close all of the system dia- 
logs that are still open. 

With these requirements taken care of, you can install the Platform.sh 
CLI. You'll need to use a Windows terminal with administrator privileges; 
enter command into the Start menu, and under the Command Prompt app, 


click Run as administrator. In the terminal that appears, enter the follow- 
ing command: 


> curl -fsS https://platform.sh/cli/installer | php 


This will install the Platform.sh CLI, as described earlier. 

Finally, you'll work in Git Bash. To open a Git Bash terminal, go to the 
Start menu and search for git bash. Click the Git Bash app that appears; 
you should see a terminal window open. You can use traditional Linux- 
based commands like 1s in this terminal, as well as Windows-based com- 
mands like dir. To make sure the installation was successful, issue the 
platform list command. You should see a list of all the commands in the 
Platform.sh CLI. From this point forward, carry out all of your deployment 
work using the Platform.sh CLI inside a Git Bash terminal window. 


Deploying from macOS 


The macOS operating system is not based on Linux, but they were both 
developed on similar principles. What this means, practically, is that a lot of 
the commands and workflows that you use on macOS will work in a remote 
server environment as well. You might need to install some developer-focused 
resources in order to have all of these tools available in your local macOS 
environment. If you get a prompt to install the command line developer tools at 
any point in your work, click Install to approve the installation. 

The most likely difficulty when installing the Platform.sh CLI is mak- 
ing sure PHP is installed. If you see a message that the php command is not 
found, you’ll need to install PHP. One of the easiest ways to install PHP is 
by using the Homebrew package manager, which facilitates the installation 
of a wide variety of packages that programmers depend on. If you don’t 
already have Homebrew installed, visit https://brew.sh and follow the instruc- 
tions to install it. 

Once Homebrew is installed, use the following command to install PHP: 


$ brew install php 


This will take a while to run, but once it has completed, you should be 
able to successfully install the Platform.sh CLI. 


Deploying from Linux 

Because most server environments are Linux-based, you should have very 
little difficulty installing and using the Platform.sh CLI. If you try to install 
the CLI on a system with a fresh installation of Ubuntu, it will tell you 
exactly which packages you need: 


$ curl -fsS https://platform.sh/cli/installer | php 
Command 'curl' not found, but can be installed with: 
sudo apt install curl 

Command 'php' not found, but can be installed with: 
sudo apt install php-cli 
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The actual output will have more information about a few other pack- 
ages that would work, plus some version information. The following com- 
mand will install curl and PHP: 


$ sudo apt install curl php-cli 


After running this command, the Platform.sh CLI installation com- 
mand should run successfully. Since your local environment is quite similar 
to most Linux-based hosting environments, much of what you learn about 
working in your terminal will carry over to working in a remote environ- 
ment as well. 


Other Deployment Approaches 
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If Platform.sh doesn’t work for you, or if you want to try a different approach, 
there are many hosting platforms to choose from. Some work similarly to the 
process described in Chapter 20, and some have a much different approach 
to carrying out the steps described at the beginning of this appendix: 


e Platform.sh allows you to use a browser to carry out the steps we used 
the CLI for. If you like browser-based interfaces better than terminal- 
based workflows, you may prefer this approach. 


e There are a number of other hosting providers that offer both CLI- and 
browser-based approaches. Some of these providers offer terminals 
within their browser, so you don’t have to install anything on your 
system. 


e Some providers allow you to push your project to a remote code hosting 
site like GitHub, and then connect your GitHub repository to the host- 
ing site. The host then pulls your code from GitHub, instead of requir- 
ing you to push your code from your local system directly to the host. 
Platform.sh supports this kind of workflow as well. 


e Some providers offer an array of services that you select from, in 
order to put together an infrastructure that works for your project. 
This typically requires you to have a deeper understanding of the 
deployment process, and what a remote server needs in order to 
serve a project. These hosts include Amazon Web Services (AWS) and 
Microsoft’s Azure platform. It can be much harder to track your costs 
in these kinds of platforms, because each service can accrue charges 
independently. 


e Many people host their projects on a virtual private server (VPS). In this 
approach, you rent a virtual server that acts just like a remote computer, 
log in to the server, install the software needed to run your project, copy 
your code over, set the right connections, and allow your server to start 
accepting requests. 


New hosting platforms and approaches appear on a regular basis; find 
one that looks appealing to you, and invest the time to learn that provider’s 
deployment process. Maintain your project long enough so that you get to 


know what works well with your provider’s approach and what doesn’t. No 
hosting platform is going to be perfect; you'll need to make an ongoing 
judgement call about whether the provider you're currently using is good 
enough for your use case. 

I'll offer one last word of caution about choosing a deployment plat- 
form and an overall approach to deployment. Some people will enthusiasti- 
cally steer you toward overly complex deployment approaches and services 
that are meant to make your project highly reliable and capable of serving 
millions of users simultaneously. Many programmers spend lots of time, 
money, and energy building out a complex deployment strategy, only to 
find that hardly anyone is using their project. Most Django projects can be 
set up on a small hosting plan and tuned to serve thousands of requests per 
minute. If your project is getting anything less than this level of traffic, take 
the time to configure your deployment to work well on a minimal platform 
before investing in infrastructure that's meant for some of the largest sites 
in the world. 

Deployment is incredibly challenging at times, but just as satisfying 
when your live project works well. Enjoy the challenge, and ask for help 
when you need it. 


Troubleshooting Deployments 501 


INDEX 


Symbols 
+ (addition), 26 
+= (addition in place), 122 
* (arbitrary arguments), 146 
** (arbitrary keyword arguments), 148 
{} (braces) 
dictionaries, 92 
sets, 104 
@ (decorator), 221, 424 
/ (division), 26 
== (equality), 72, 74 
** (exponent), 26 
> (greater than), 75 
>= (greater than or equal to), 75 
# (hash mark), for comments, 29 
l= (inequality), 74 
< (less than), 75 
<= (less than or equal to), 75 
[] (list), 34 
% (modulo), 116 
+= (multiline strings), 115 
* (multiplication), 26 
\n (newline), 22 
>>> (Python prompt), 4 
- (subtraction), 26 
\t (tab), 22 
_ (underscore) 
in file and folder names, 10 
in numbers, 28 
in variable names, 17 


A 


aliases, 151-152, 178-179 
alice.py, 195-197 
Alien Invasion. See also Pygame 
aliens, 256-274 
building fleet, 259-262 
checking edges, 265 


collisions, with bullets, 267 
collisions, with ship, 270-273 
controlling fleet direction, 
264-266 
creating an alien, 256-258 
dropping fleet, 265—266 
reaching bottom of screen, 
273-274 
rebuilding fleet, 268-269 
bullets, 247-253, 266—270 
collisions, with aliens, 267 
deleting old, 250—251 
firing, 249-250 
larger, 268 
limiting number of, 251-252 
settings, 247 
speeding up, 269 
classes 
Alien, 257 
AlienInvasion, 229 
Bullet, 247—248 
Button, 278-279 
GameStats, 271 
Scoreboard, 286-287 
Settings, 232 
Ship, 234-235 
ending the game, 274—275 
initializing dynamic settings, 
283-285 
levels 
modifying speed settings, 
283-285 
resetting the speed, 285 
displaying, 294—296 
moving fleet, 263—266 
planning, 228 
Play button, 278-283 
Button class, 278—279 
deactivating, 282 
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Alien Invasion (continued) 
Play button (continued) 
drawing, 279-280 


hiding mouse cursor, 282-283 


resetting game, 281-282 
starting game, 281 
reviewing the project, 256 
scoring, 286-298 
all hits, 290 
high score, 292-294 
increasing point values, 
290—291 
level, 294—296 
number of ships, 296—299 
resetting, 289-290 
rounding and formatting, 
291-292 
score attribute, 286 
updating, 289 
settings, storing, 232-233 
ship, 233—244 
adjusting speed, 242-243 
continuous movement, 
239-242 
finding an image, 233-234 
limiting range, 243—244 
amusement, park.py, 80—82 
and keyword, 75 
antialiasing, 279 
API. See application programming 
interface 
apostrophe.py, 24—25 
append() method, 37-38 
application programming 
interface (API), 355 
API call, 355-357 
GitHub API, 368 
Hacker News API, 368-371 
processing an API response, 
357-362 
rate limits, 362 
requesting data, 356-357 
visualizing results, 362-368 


arguments, 131. See also under functions 


as keyword, 151-152 
assertions, 213, 217-218 
attributes, 159. See also under classes 


banned, users.py, 76-77 
bicycles.py, 34—35 
Boolean values, 77 
Bootstrap, 433. See also unxder Django 
braces ({}) 
dictionaries, 92 
sets, 104 
break statement, 121 
built-in functions, 467 
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calls (functions), 130, 132-135 
car.py, 162-178 
cars.py, 43-45, 72 
cities.py, 121 
classes 
attributes, 159 
accessing, 160 
default values, 163-164 
modifying, 164-166 
creating, 158-161 
importing, 173-179 
multiple classes, 175-176 
single classes, 174-175 
inheritance, 167-172 
attributes and methods, 169 
child classes, 167-170 
composition, 170 
. init () method, 167-169 
instances as attributes, 170—172 
overriding methods, 170 
parent classes, 170 
subclasses, 168 
super() function, 168 
superclasses, 168 
instances, 157 
methods, 159 
calling, 160 
chaining, 185 
. init() method, 159 
modeling real-world objects, 
172-173 
multiple instances, 161 
naming conventions, 158 
objects, 157 
style guidelines, 181 


comma-separated value files. See CSV files 


comment.py, 29 
comments, 29-30 
conditional tests, 72-77. See also 
if statements 
confirmed_users.py, 124-125 
constants, 28 
continue statement, 122 
counting.py, 117-118, 122-123 
CSV files, 330-341 
csv.reader() function, 330-333 
error checking, 338-341 
file headers, 330—332 


D 
data analysis, 301 
databases. See under Django 
data visualization, 301. See also 
Matplotlib; Plotly 
datetime module, 333-335 
death_valley_highs_lows.py, 339-341 
decorators, 221—223, 423—425 
default values 
class attributes, 163—164 
function parameters, 134-135 
definition (functions), 130 
def keyword, 130 
del statement 
with dictionaries, 96 
with lists, 38-40 
dice_visual_d6d10.py, 326-327 
dice_visual.py, 324—326 
dictionaries 
defining, 92 
empty, 94 
formatting larger, 96-97 
KeyError, 98 
key-value pairs, 92 
adding, 93-94 
removing, 96 
looping through 
keys, 101-102 
keys in order, 102-103 
key-value pairs, 99-101 
values, 103-104 
methods 
get(), 97-98 
items(), 99-101 


keys(), 101-103 
values(), 103-104 
nesting 
dictionaries in dictionaries, 
110-111 
dictionaries in lists, 105-108 
lists in dictionaries, 108—109 
ordering in, 94, 102-103 
sorting a list of, 370 
values 
accessing, 92-93, 97-98 
modifying, 94-96 
die.py, 320 
die visual.py, 320—321 
dimensions.py, 66-67 
div (HTML), 437 
division_calculator.py, 192-195 
Django. See also Git; Learning Log project 
accounts app, 415—423 
creating app, 415-416 
logging out, 419-420 
login page, 416-419 
registration page, 420—423 
admin site, 381-386 
associating data with a user, 
425—430 
Bootstrap, 434—445 
card, 443 
collapsible navigation, 437 
container, 440 
django-boostrap5 app, 434 
documentation, 444 
HTML headers, 435—436 
jumbotron, 440—441 
list groups, 443 
navigation bar, 436—439 
styling forms, 441-442 
commands 
createsuperuser, 382 
flush, 427 
makemigrations, 381, 385, 426 
migrate, 377 
runserver, 377—378, 383, 392 
shell, 386 
startapp, 379, 415 
startproject, 376 
creating new projects, 376 
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databases 
cascading delete, 384 
creating, 376 
foreign keys, 384, 425 


many-to-one relationships, 384 


migrating, 377, 381, 385, 426 

non-nullable field, 427 

Postgres, 447 

queries, 398, 428 

querysets, 386—387, 395, 398, 
426—428 

resetting, 427 

SQLite, 377 

deployment, 445—461, 493—501 

committing the project, 453 

configuration files, 447—450 

creating Platform.sh project, 
453-455 

creating superuser, 456-457 

custom error pages, 459—460 

deleting projects, 461 

free trial limits, 446 

gunicorn, 447 

ignoring files, 452-453 

installing Platform.sh CLI, 
446, 497-500 

installing platform 
shconfig, 446 

other deployment 
approaches, 500 

Platform.sh, 445 

Postgres database, 447, 
450—451 

psycopg2, 447 

pushing a project, 455 

pushing changes, 458, 460 

requirements.txt, 446 

securing project, 457-460 

settings, 451 

SSH sessions, 456—457 

troubleshooting, 494—501 

using Git, 451 

viewing project, 456 

development server, 377-378, 
383, 392 


documentation 
model fields, 380 
queries, 388 
templates, 400 
forms, 404—423, 429-430 
csrf token, 407 
GET and POST requests, 406 
ModelForm, 404, 408 
processing forms, 405—406, 
409—410, 412-413, 
421—422, 429—430 
save() method, 405—406, 
409—410, 430 
templates, 407, 410—411, 413, 
417, 419, 422 
validation, 404—406 
widgets, 408 
HTML 
anchor tag (<a>), 393 
«body» element, 437 
comments, 437 
«div» elements, 437 
«main? element, 440 
margins, 440 
padding, 440 
«p» elements, 391 
«span? elements, 438 
HTTP 404 error, 428—429, 459—460 
INSTALLED APPS, 380 
installing, 375-376 
localhost, 378 
logging out, 419—420 
Glogin required decorator, 423—424 
login template, 417 
mapping URLs, 388-390, 397-398 
migrating the database, 426—427 
models, 379 
activating, 380—381 
defining, 379, 384 
foreign keys, 384, 425 
registering with admin, 
382-383, 385-386 
. Str () method, 380, 384 
projects (vs. apps), 379 
redirect() function, 405—406 
release cycle, 376 
restricting access to data, 427—430 


settings 
ALLOWED_HOSTS, 451 
DEBUG, 457—458 
INSTALLED APPS, 380—381, 
415—416, 434 
LOGIN REDIRECT URL, 417-418 
LOGIN URL, 424 
LOGOUT REDIRECT URL, 420 
SECRET KEY, 451 
shell, 386—387, 426—427 
starting an app, 379 
styling. See Django: Bootstrap 
superusers, 382, 456—457 
templates 
block tags, 398 
child template, 393-394 
context dictionary, 395 
filters, 399 
forms in, 407 
indentation in, 393 
inheritance, 392—394 
linebreaks, 399 
links in, 392-393, 399 
loops in, 395-397 
parent template, 392-393 
template tags, 393 
timestamps in, 398-399 
user object, 418 
writing, 390-392 
URLs. See Django: mapping URLs 
UserCreationForm, 421—422 
user ID values, 426 
versions, 376 
view functions, 388, 390 
virtual environments, 374—375 
docstrings, 130, 153, 181 
dog.py, 158-162 
dot notation, 150, 160 


earthquakes. See mapping earthquakes 
electric car.py, 167-173 
module, 177-179 
encoding argument, 195-196 
enumerate() function, 331 
eq explore data.py, 348—347 
equality operator (==), 72, 74 
eq world, map.py, 347-352 


even, mumbers.py, 58 

even, or odd.py, 117 

exceptions, 183, 192-199 
deciding which errors to report, 199 
else block, 194-195 
failing silently, 198-199 
FileNotFound error, 195-196 
handling exceptions, 192-196 
preventing crashes, 193-195 
try-except blocks, 193 
ZeroDivisionError, 192-195 

exponents (**), 26 


F 
favorite. languages.py, 96-97, 100—104, 109 
file. reader.py, 184-187 
files 
encoding argument, 195-196 
FileNotFound error, 195-196 
file paths, 186 
absolute, 186 
exists() method, 203-204 
pathlib module, 184 
Path objects, 184-186, 330 
relative, 186 
from strings, 198 
on Windows, 186 
read text() method, 185, 195-196 
splitlines() method, 186-187 
write text() method, 190-191 
first. mumbers.py, 57 
fixtures, 221-223 
flags, 120-121 
floats, 26—28 
foods.py, 63—64 
for loops, 49-56, 99-104. See also 
dictionaries; lists 
formatted, name.py, 137-139 
Fstrings 
format specifiers, 201—292 
using variables in, 20-21 
full_name.py, 21 
functions, 129-155 
arguments 
arbitrary, 146—149 
default values, 134—135 
errors, 136 
keyword, 133-134 
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functions (continued) installing, 484 


arguments (continued) log, 487 
lists as, 142-145 repositories, 356 
optional, 138-139 status, 485—486 
positional, 131-133 GitHub, 356 
body, 130 greeter.py, 114—115, 130-131 
built-in, 467 greet_users.py, 142 
calling functions, 130, 132-135 
defining, 130 H 
importing, 149-153 Hacker News API, 368-371 
aliases, 151-152 hash mark (#), for comments, 29 
entire modules, 150-151 hello_git.py, 484—491 
specific functions, 151 hello_world.py, 10-12, 15-19 
modifying a list in a function, hidden files, 448, 485 
142-145 hn_article.py, 368-369 
modules, 149-153 hn_submissions.py, 369-371 
parameters, 131 
return values, 137-141 l 
style guidelines, 153 IDE (integrated development 
environment), 469—470 
G if statements 
GeoJSON files, 342-347, 350-351 and keyword, 75 
GET requests, 406. See Django: forms Boolean expressions, 77 
getting help checking for 
Discord, 480 equality (==), 72 
official Python documentation, inequality (!=), 74 
479—480 item in list, 76 
online resources, xxxv, 478 item not in list, 76 
r/learnpython, 480 list not empty, 86-87 
rubber duck debugging, 478 elif statement, 80-83 
searching online, 479 else statement, 79-80 
Slack, 481 if statements and lists, 85—88 
Stack Overflow, 479 ignoring case, 73-74 
three main questions, 477—478 numerical comparisons, 74—76 
Git, 356, 451—453, 483-492. See also or keyword, 76 
Django: deployment simple, 78 
abandoning changes, 488—489 style guidelines, 89 
adding files, 486 testing multiple conditions, 82-83 
branches, 486 immutable, 65 
checking out previous commits, import *, 152, 177 
489-491 import this, 30-31 
commits, 486—488 indentation errors, 53—56 
configuring, 452, 484 index errors, 46—47 
deleting a repository, 491—492 inheritance, 167—173. See also 
.gitignore, 484 under classes 
HEAD, 490 input() function, 114-116 
ignoring files, 484 numerical input, 115-116 
initializing a repository, 485 writing prompts, 114-115 


508 Index 


insert() method, 38 
itemgetter() function, 370 
items() method, 99-101 


J 


JSON files 
GeoJSON files, 342-347, 350-351 
JSON data format, 201 
json.dumps() function, 201-204, 

343-344, 368 
json. loads() function, 201-204, 
343-344 


K 

keys() method, 101-103 

key-value pairs, 92. See also dictionaries 
keyword arguments, 133-134 
keywords, 466 


L 


language survey.py, 219 
Learning Log project, 373 
files, 392 

404. html, 459 

500.html, 459 

accounts/urls.py, 416, 420 

accounts/views.py, 421-422 

admin.py, 382-383 

base.himl, 392-393, 396, 
418-419, 422, 435-440 

edit_entry.himl, 413 

forms.py, 404, 408—409 

.gitignore, 452—453 

index.html, 390—394, 440—441 

learning. logs/urls.py, 389—390, 
394—395, 397-398, 405, 
409, 412 

learning. logs/views.py, 390, 
395, 398, 405—406, 409- 
410, 412—418, 423-425, 
428—430 

Il. project/urls.py, 388—389, 416 

login.html, 417, 441-442 

models.py, 379-380, 384 

new_entry.himl, 410 

new_topic.himl, 407 

.platform.app.yaml, 448—450 


register.html, 422 
requirements.txt, 446—447 
routes.yaml, 450 
services.yaml, 450 
settings.py, 380—381, 415—418, 
420, 424, 434, 451, 
457-460 
topic.himl, 398—399, 443—444 
topics.html, 395-396, 442-448 
ongoing development, 460 
pages, 391 
edit entry, 412-414 
home page, 388-394 
login page, 416-419 
new entry, 408-411 
new topic, 404—408 
registration, 420—423 
topic, 397-400 
topics, 394—397 
writing a specification (spec), 374 
len() function, 44—45 
library, 184 
Linux 
Python 
checking installed version, 8 
setting up, 8-12, 465—466 
terminals 
running programs from, 12 
starting Python session, 9 
troubleshooting installation 
issues, 10 
VS Code, installing, 9 
lists, 33 
as arguments, 142-145 
comprehensions, 59-60 
copying, 63-64 
elements 
accessing, 34 
accessing last, 35 
adding with append(), 37-38 
adding with insert(), 38 
identifying unique, 104 
modifying, 36-37 
removing with del, 38-39 
removing with pop(), 39-40 
removing with remove(), 40-41 
empty, 37-38 
enumerate() function, 331 
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lists (continued) 
errors 
indentation, 53-56 
index, 46 
for loops, 49-56 
nested, 108-109, 261-262 
indexes, 34-35 
negative index, 35 
zero index, 34-35 
len() function, 44-45 
naming, 33-34 
nesting 
dictionaries in lists, 105—108 
lists in dictionaries, 108—109 
numerical lists, 56-60 
max() function, 59 
min() function, 59 
range() function, 58-59 
sum() function, 59 
removing all occurrences ofa 


value, 125 
slices, 61-62 
sorting 


reverse() method, 44 
sorted() function, 43-44 
sort() method, 43 
square brackets, 34 
logical errors, 54 
lstrip() method, 22-23 


M 
macOS 
.DS Store files, ignoring, 453 
Homebrew package manager, 499 
Python 
checking installed version, 7 
setting up, 7-12, 464—465 
terminals 
running programs from, 12 
starting Python session, 7 
troubleshooting installation 
issues, 10 
VS Code, installing, 8 
magicians.py, 49—56 
magic mumber.py, 74 
making pizzas.py, 150—152 
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mapping earthquakes, 342—352. See 
also Plotly 
downloading data, 343, 352 
GeoJSON files, 342-347, 350—351 
latitude-longitude ordering, 345 
location data, 346-347 
magnitudes, 346 
world map, 347-348 
Matplotlib 
axes 
set aspect() method, 313-314 
removing, 317 
ax objects, 303 
colormaps, 310-311 
fig objects, 308 
figsize argument, 318 
formatting plots 
alpha argument, 337-338 
built-in styles, 306 
custom colors, 310 
labels, 303—304 
line thickness, 303-304 
plot size, 318 
shading, 337-338 
tick labels, 309—310 
gallery, 302 
installing, 302 
plot() method, 303-306 
pyplot module, 302-303 
savefig() method, 311 
saving plots, 311 
scatter() method, 306-311 
simple line graph, 302-306 
subplots() function, 303 
methods, 20 
helper methods, 237 
modules, 149-152, 173—179. See also 
classes: importing; functions: 
importing 
modulo operator (4), 116-117 
motorcycles.py, 36—41 
mountain, poll.py, 125-126 
mpl. squares.py, 302-306 
my car.py, 174—175 
my. cars.py, 176-179 
my. electric. car.py, 176 


name errors, 17-18 
name_function.py, 211—217 
name.py, 20 
names.py, 211-212 
nesting. See dictionaries: nesting; lists: 
for loops 
newline (\n), 21-22 
next() function, 330-331 
None, 98, 140 
number_reader.py, 202 
numbers, 26-28 
arithmetic, 26 
constants, 28 
exponents, 26 
floats, 26-27 
formatting, 291—292 
integers, 26 
mixing integers and floats, 27-28 
order of operations, 26 
round() function, 291-292 
underscores in, 28 
number wriler.py, 201 


0 


object-oriented programming 
(OOP), 157. See also classes 
or keyword, 76. See also if statements 


P 


pandas, 320 

parameters, 131 

parrot.py, 114, 118-121 

pass statement, 198-199 

paths. See files: file paths 

PEP 8, 68-69 

person.py, 139—140 

pets.py, 125, 132-136 

pip, 210-211 
installing Django, 374-376 
installing Matplotlib, 302 
installing Plotly, 320 
installing Pygame, 228 
installing pytest, 211 
installing Requests, 357 
Linux, installing pip, 465—466 
updating, 210 


pi string.py, 187-189 
pizza.py, 146—148 
Platform.sh. See Django: deployment 
players.py, 61-62 
Plotly, 302, 319. See also mapping 
earthquakes; rolling dice 
chart types, 322 
customizing plots, 323, 325-326, 364 
documentation, 368 
fig.show() method, 322 
fig.write html() method, 327 
formatting plots 
axis labels, 323 
color scales, 349—350 
hover text, 350-351, 365-366 
links in charts, 366—367 
marker colors, 349—350, 367 
tick marks, 325-326 
titles, 323 
tooltips, 365-366 
update layout() method, 
325-326, 364 
update_traces() method, 367 
gallery, 320 
histograms, 322 
installing, 320 
plotly.express module, 322, 
347, 368 
px alias, 322 
px.bar() function, 322—323, 363-367 
saving figures, 327 
scatter geo() function, 347-352 
pop() method, 39-40 
positional arguments, 131—133. See also 
functions: arguments 
POST requests, 406. See also 
Django: forms 
printing_models.py, 143-145 
Project Gutenberg, 196-197 
prompts, 114-115 
.py file extension, 15-16 
Pygame. See also Alien Invasion 
background colors, 231—232 
clock.tick() method, 230-231 
collisions, 266—267, 270-271, 
289-290 
creating an empty window, 229-230 
cursor, hiding, 282-283 
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Pygame (continued) 
displaying text, 278-280 
ending games, 274-275 
event loops, 229-230 
frame rates, 230-231 
fullscreen mode, 245 
groups 
adding elements, 249—250 
defining, 248—249 
drawing all elements in, 
249-250, 257-258 
emptying, 268-269 
looping through, 249—251 
removing elements from, 
250-251 
updating all elements in, 
248-249 
images, 234—236 
installing, 228 
levels, 288-285 
Play button, 278-283 
print() calls in, 251 
quitting, 244—245 
rect objects, 234—235 
creating from scratch, 
247—248 
get rect() method, 234—235 
positioning, 234—235, 238-243, 
247-248, 256-262, 278, 
286-298 
size attribute, 261 
responding to input, 230 
events, 230 
keypresses, 238-242 
mouse clicks, 281-283 
screen coordinates, 235 
surfaces, 230 
testing games, 268 
pytest. Seetesting code 
Python 
>>> prompt, 4 
built-in functions, 467 
checking installed version, 466 
installing 
on Linux, 465—466 
on macOS, 7-11, 464—465 
on Windows, 5—6, 463—464 


interpreter, 15-16 
keywords, 466 
Python Enhancement Proposal 
(PEP), 68 
standard library, 179-180 
terminal sessions, 4 
on Linux, 9 
on macOS, 7-8 
on Windows, 6 
versions, 4 
why use Python, xxxvi 
python, repos.py, 357-362 
python. repos visual.py, 362-367 


Q 


quit values, 118 


random, walk.py, 312—313 
random walks, 312-318 
choice() function, 313 
coloring points, 315-316 
fill_walk() method, 312-313 
generating multiple walks, 314—315 
plotting, 313-314 
RandomWalk class, 312-313 
starting and ending points, 316-317 
range() function, 58—59 
read text() method, 185, 195-196 
refactoring, 204—206, 237—238, 260, 
269-270 
remember me.py, 202—206 
removeprefix() method, 24 
removesuffix() method, 25 
Requests package, installing, 357 
return values, 137-141 
rollercoaster.py, 116 
rolling dice, 319—327. See also Plotly 
analyzing results, 321-322 
Die class, 320 
different-size dice, 326-327 
randint() function, 320 
rolling two dice, 324—326 
rubber duck debugging, 478 
rstrip() method, 22-23 
rw_visual.py, 313-318 


$ 


scatter squares.py, 306—311 
sets, 103-104 
sitka, highs lows.py, 336—338 
sitka, highs.py, 330—336 
sleep() function, 272 
slices, 61-64 
sorted() function, 43—44, 102-103 
sort() method, 43 
splitlines() method, 186-187 
split() method, 196-197 
SOLite database, 376-377 
square mwumbers.py, 58-59 
squares.py, 59—60 
Stack Overflow, 479 
storing data, 201—204. See also JSON files 
saving and reading data, 202-204 
strings, 19-25 
changing case, 20 
fstrings, 20—21, 291-292 
methods 
lower(), 20 
lstrip(), 22-23 
removeprefix(), 23-24 
removesuffix(), 25 
rstrip(), 22-23 
split(), 196-197 
splitlines(), 186-187 
strip(), 22-23 
title(), 20 
upper(), 20 
multiline, 115 
newlines in, 21-22 
single and double quotes, 19, 24-25 
tabs in, 21-22 
variables in, 20-21 
whitespace in, 21-23 
strip() method, 22-23 
strptime() method, 333-335 
style guidelines, 68-69 
blank lines, 69 
CamelCase, 181 
classes, 181 
dictionaries, 96-97 
functions, 153 
if statements, 89 
indentation, 68 


line length, 69 

PEP 8, 68 
survey.py, 218 
syntax errors, 24 

avoiding with strings, 24—25 
syntax highlighting, 16 


T 
tab (Mt), 21-22 
templates. See under Django 
testing code, 209—223 
assertions, 213, 217-218 
failing tests, 214—216 
full coverage, 212 
naming tests, 213 
passing tests, 212-214 
pytest, 209—223 
fixtures, 221-223 
installing, 210-211 
running tests, 213-214 
test cases, 212 
testing classes, 217-223 
testing functions, 211-217 
unit tests, 212 
test name function.py, 212-217 
lest, survey.py, 220—223 
text editors and IDEs. See also VS Code 
Emacs and Vim, 475 
Geany, 474 
IDLE, 474 
Jupyter Notebooks, 475 
PyCharm, 475 
Sublime Text, 474 
third-party package, 210 
toppings.py, 74, 82-83 
tracebacks, 10, 17-18, 192, 195-196 
try-except blocks. See exceptions 
tuples, 65-67 
defining, 65 
for loop, 66-67 
writing over, 67 
type errors, 66 


U 


underscore (_) 
in file and folder names, 10 
in numbers, 28 
in variable names, 17 
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unit tests, 212 
user profile.py, 148—149 


V 


values() method, 103-104 
variables, 16-19, 28 
constants, 28 
as labels, 18-19 
multiple assignment, 28 
name errors, 17-18 
naming conventions, 17 
values, 16 
venv module, 374—375 
version control. See Git 
virtual environments, 374—375 
voting.py, 78-80 
VS Code, 4-5 
configuring, 470—478 
features, 469—470 
installing 
on Linux, 9 
on macOS, 8 
on Windows, 6 
Python extension, 9-10 
opening files with Python, 185 
Python extension, 9 
running files, 10 
shortcuts, 473—474 
tabs and spaces, 471 


W 


weather data, 330—341. See also CSV 
files; Matplotlib 
while loops, 117-126 
active flag, 120-121 
break statement, 121 
continue statement, 122 
infinite loops, 122-123 
moving items between lists, 124 
quit values, 118 
removing all items from list, 125 
whitespace, 21—23. See also strings 
Windows 
file paths, 186 
Python 
setting up, 5-6, 9-12, 463—464 
troubleshooting installation, 10 
terminals 
running programs from, 12 
starting Python session, 6 
VS Code, installing, 6 
word, count.py, 197-199 
write message.py, 190—191 
write text() method, 190-191 


Z 
Zen of Python, 30-31 
ZeroDivisionError, 192-195 
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